Jul 02 23:08:21.202834 ip-10-0-136-84 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 02 23:08:21.202849 ip-10-0-136-84 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 02 23:08:21.202859 ip-10-0-136-84 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 02 23:08:21.203212 ip-10-0-136-84 systemd[1]: Failed to start Kubernetes Kubelet. Jul 02 23:08:31.395887 ip-10-0-136-84 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 02 23:08:31.395907 ip-10-0-136-84 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a643cac6aa1a4a46a6bc370fd685f11a -- Jul 02 23:10:07.881533 ip-10-0-136-84 systemd[1]: Starting Kubernetes Kubelet... Jul 02 23:10:08.324140 ip-10-0-136-84 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:10:08.324140 ip-10-0-136-84 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 23:10:08.324140 ip-10-0-136-84 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:10:08.324140 ip-10-0-136-84 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 23:10:08.324140 ip-10-0-136-84 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 23:10:08.324140 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.318547 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328325 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328343 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328347 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328350 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328353 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328356 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328359 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328362 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328365 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328367 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328370 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328373 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328376 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328379 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328382 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328385 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328388 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328392 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:08.332206 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328396 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328400 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328404 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328408 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328411 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328414 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328417 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328420 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328423 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328425 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328428 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328430 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328433 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328443 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328446 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328448 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328451 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328454 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328457 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:08.346882 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328459 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328462 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328465 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328467 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328470 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328473 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328477 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328480 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328483 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328486 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328489 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328492 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328494 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328497 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328500 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328503 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328505 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328508 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328511 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:08.348665 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328513 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328517 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328520 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328522 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328525 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328527 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328530 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328532 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328535 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328538 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328541 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328543 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328547 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328550 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328553 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328555 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328558 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328560 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328563 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328565 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:08.351007 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328569 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328572 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328575 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328578 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328581 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328584 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328587 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328590 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328594 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328597 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328599 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328602 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328605 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328607 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328610 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328613 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328615 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328618 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328621 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328623 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:08.362538 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328626 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328629 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.328632 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330443 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330453 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330457 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330459 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330463 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330466 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330469 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330471 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330474 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330478 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330480 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330483 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330486 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330489 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330491 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330494 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330497 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:08.364719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330500 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330503 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330506 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330509 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330512 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330514 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330517 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330520 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330523 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330526 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330528 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330532 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330537 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330540 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330542 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330545 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330547 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330550 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330553 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:08.366622 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330556 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330560 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330563 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330566 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330569 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330571 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330576 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330579 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330582 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330585 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330588 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330591 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330593 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330596 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330599 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330601 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330604 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330607 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330609 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:08.379035 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330612 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330614 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330617 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330619 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330624 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330627 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330630 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330633 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330636 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330639 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330642 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330645 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330648 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330651 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330653 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330656 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330659 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330661 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330664 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330670 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:08.429034 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330672 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:08.399381 ip-10-0-136-84 systemd[1]: Started Kubernetes Kubelet. Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330675 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330678 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330680 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330683 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330686 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330688 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330691 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330695 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330697 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330700 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330704 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330706 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330709 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330712 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330714 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330717 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330719 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330722 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330725 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:08.460957 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330728 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330730 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330733 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.330735 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331124 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331129 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331132 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331134 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331137 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331140 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331143 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331147 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331150 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331153 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331155 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331159 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331162 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331164 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331167 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331206 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:08.674862 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331210 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331213 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331216 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331219 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331223 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331227 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331230 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331233 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331236 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331239 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331242 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331245 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331247 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331250 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331252 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331255 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331258 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331260 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331263 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:08.675719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331265 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331268 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331270 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331273 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331277 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331279 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331282 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331284 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331287 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331290 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331293 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331295 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331299 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331302 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331305 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331307 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331310 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331313 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331315 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331318 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:08.676347 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331320 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331324 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331327 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331329 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331332 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331335 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331339 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331342 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331345 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331348 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331350 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331353 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331356 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331359 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331362 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331365 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331369 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331372 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331375 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331377 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:08.676975 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331381 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331384 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331386 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331389 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331393 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331395 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331398 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331401 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331404 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331406 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331409 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331412 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331416 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331419 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331422 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331424 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331427 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331429 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331431 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331434 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:08.677619 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331436 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331439 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331441 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.331444 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331509 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331516 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331523 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331530 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331538 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331558 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331565 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331570 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331573 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331576 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331580 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331583 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331589 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331592 2570 flags.go:64] FLAG: --cgroup-root="" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331595 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331598 2570 flags.go:64] FLAG: --client-ca-file="" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331602 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331605 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331609 2570 flags.go:64] FLAG: --cluster-domain="" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331612 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 23:10:08.678272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331615 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331618 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331621 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331625 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331628 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331631 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331634 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331637 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331640 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331643 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331648 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331650 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331654 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331658 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331661 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331663 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331668 2570 flags.go:64] FLAG: --enable-server="true" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331671 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331676 2570 flags.go:64] FLAG: --event-burst="100" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331679 2570 flags.go:64] FLAG: --event-qps="50" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331682 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331685 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331688 2570 flags.go:64] FLAG: --eviction-hard="" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331692 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 23:10:08.678943 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331698 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331701 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331704 2570 flags.go:64] FLAG: --eviction-soft="" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331707 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331710 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331713 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331716 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331720 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331723 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331725 2570 flags.go:64] FLAG: --feature-gates="" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331729 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331732 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331735 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331739 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331742 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331745 2570 flags.go:64] FLAG: --help="false" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331748 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331751 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331754 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331757 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331760 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331763 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331766 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 23:10:08.679690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331769 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331774 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331777 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331779 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331783 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331785 2570 flags.go:64] FLAG: --kube-reserved="" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331788 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331791 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331794 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331799 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331802 2570 flags.go:64] FLAG: --lock-file="" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331804 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331807 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331810 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331816 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331819 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331823 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331826 2570 flags.go:64] FLAG: --logging-format="text" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331829 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331832 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331835 2570 flags.go:64] FLAG: --manifest-url="" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331838 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331842 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331845 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331849 2570 flags.go:64] FLAG: --max-pods="110" Jul 02 23:10:08.680501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331852 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331855 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331857 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331860 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331863 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331866 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331869 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331877 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331881 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331885 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331887 2570 flags.go:64] FLAG: --pod-cidr="" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331890 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331897 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331900 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331903 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331905 2570 flags.go:64] FLAG: --port="10250" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331910 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331913 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aa6bf975021b947f" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331917 2570 flags.go:64] FLAG: --qos-reserved="" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331919 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331922 2570 flags.go:64] FLAG: --register-node="true" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331926 2570 flags.go:64] FLAG: --register-with-taints="" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331929 2570 flags.go:64] FLAG: --registry-burst="10" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331933 2570 flags.go:64] FLAG: --registry-qps="5" Jul 02 23:10:08.681311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331935 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331939 2570 flags.go:64] FLAG: --reserved-memory="" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331942 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331946 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331948 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331951 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331954 2570 flags.go:64] FLAG: --runonce="false" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331957 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331960 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331963 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331965 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331968 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331972 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331975 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331978 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331980 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331983 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331988 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331991 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331994 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331996 2570 flags.go:64] FLAG: --system-cgroups="" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.331999 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332005 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332007 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332011 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 23:10:08.682009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332015 2570 flags.go:64] FLAG: --tls-min-version="" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332018 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332021 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332024 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332027 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332030 2570 flags.go:64] FLAG: --v="2" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332033 2570 flags.go:64] FLAG: --version="false" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332038 2570 flags.go:64] FLAG: --vmodule="" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332042 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.332045 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332166 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332171 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332174 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332177 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332195 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332198 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332201 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332204 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332209 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332213 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332216 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332218 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:08.682762 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332221 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332224 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332228 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332231 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332233 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332236 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332238 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332241 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332244 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332248 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332251 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332254 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332258 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332262 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332265 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332267 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332270 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332273 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332276 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:08.683419 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332279 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332282 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332285 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332288 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332291 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332293 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332296 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332299 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332301 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332304 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332307 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332310 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332312 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332315 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332317 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332320 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332322 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332325 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332328 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:08.683963 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332331 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332333 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332336 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332338 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332341 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332345 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332350 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332353 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332356 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332359 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332362 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332365 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332367 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332370 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332373 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332376 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332379 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332381 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332384 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:08.684492 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332386 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332389 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332392 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332394 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332397 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332400 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332402 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332404 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332407 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332410 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332412 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332415 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332418 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332420 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332423 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332426 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332428 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332430 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332433 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332437 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:08.685076 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332440 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332442 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332445 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332447 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332451 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332453 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332456 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332460 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332463 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.332465 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.333238 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.340411 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.340427 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.340443 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.340449 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 23:10:08.685713 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340525 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340530 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340533 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340536 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340541 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340544 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340546 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340550 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340552 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340555 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340558 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340560 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340563 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340566 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340568 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340571 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340574 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340578 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340581 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340584 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:08.686565 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340587 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340589 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340592 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340595 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340598 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340601 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340605 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340609 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340613 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340616 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340619 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340622 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340624 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340627 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340630 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340633 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340637 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340640 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340643 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340645 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:08.687437 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340648 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340651 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340654 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340656 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340659 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340661 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340664 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340667 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340669 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340675 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340677 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340680 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340683 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340685 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340688 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340691 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340693 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340696 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340699 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340701 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:08.688064 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340704 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340706 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340709 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340711 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340714 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340716 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340719 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340722 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340726 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340728 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340731 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340733 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340736 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340739 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340742 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340745 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340747 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340750 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340754 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340757 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:08.690746 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340760 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340764 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340767 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340769 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340772 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340774 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340777 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340780 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340783 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340785 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340790 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340793 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340796 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340799 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340802 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340805 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340807 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340810 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:08.691361 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340813 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.340817 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340932 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340937 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340940 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340944 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340948 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340951 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340954 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340957 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340960 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340963 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340967 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340970 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 23:10:08.854689 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340973 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340976 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340980 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340983 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340986 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340988 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340991 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340994 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340996 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.340999 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341001 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341004 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341007 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341009 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341012 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341015 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341017 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341020 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341022 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341025 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 23:10:08.855267 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341029 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341031 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341034 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341036 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341039 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341041 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341044 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341046 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341049 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341052 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341055 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341057 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341060 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341062 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341066 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341069 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341071 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341074 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341076 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341079 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 23:10:08.855861 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341081 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341084 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341087 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341089 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341093 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341095 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341098 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341100 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341103 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341106 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341109 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341112 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341117 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341120 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341123 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341125 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341128 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341130 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341133 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 23:10:08.856466 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341135 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341138 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341140 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341143 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341145 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341148 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341150 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341154 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341157 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341159 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341162 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341165 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341167 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341170 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341172 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341175 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341178 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341202 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341206 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341209 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 23:10:08.856979 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341212 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341215 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341217 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341220 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341222 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341226 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341229 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:08.341232 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.341237 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.341355 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.343559 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.345124 2570 server.go:1019] "Starting client certificate rotation" Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.345230 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:10:08.857583 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.345266 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 23:10:08.857966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.348850 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:10:08.857966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.351106 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 23:10:08.857966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.362469 2570 log.go:25] "Validated CRI v1 runtime API" Jul 02 23:10:08.857966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.366415 2570 log.go:25] "Validated CRI v1 image API" Jul 02 23:10:08.857966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.367543 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 23:10:08.857966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.370202 2570 fs.go:135] Filesystem UUIDs: map[39236c03-73c4-4e81-a629-d01c61d243df:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bf7c22e8-b2cf-47de-9ec4-d1a13e09ba33:/dev/nvme0n1p3] Jul 02 23:10:08.857966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.370224 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 23:10:08.857966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.374538 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.375440 2570 manager.go:216] Machine: {Timestamp:2026-07-02 23:10:08.373994506 +0000 UTC m=+0.369242787 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103821 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c3ccd8dcd54fbdd29109b32e4276b SystemUUID:ec2c3ccd-8dcd-54fb-dd29-109b32e4276b BootID:a643cac6-aa1a-4a46-a6bc-370fd685f11a Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8f:c6:ca:b7:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8f:c6:ca:b7:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:71:ea:22:6d:d6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.375547 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.375633 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.378874 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.378900 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-84.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.379779 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.379790 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.379798 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.380995 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.382696 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386323 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386434 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386464 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386473 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386489 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386498 2570 kubelet.go:398] "Adding apiserver pod source" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386507 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 23:10:08.858318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386903 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.386919 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.389434 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.392136 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.392163 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393679 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393696 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393704 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393712 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393724 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393732 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393743 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393750 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393760 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393767 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393779 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.393787 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.394681 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.394698 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.396576 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-84.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.397136 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.397208 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.398582 2570 server.go:1268] "Started kubelet" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.398627 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.399396 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.399706 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.401535 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.401702 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.404222 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.409397 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.409421 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.410065 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.411005 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.411022 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.411033 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.411418 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.411451 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.405716 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-84.ec2.internal.18be9bcd441b5630 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-84.ec2.internal,UID:ip-10-0-136-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-84.ec2.internal,},FirstTimestamp:2026-07-02 23:10:08.398538288 +0000 UTC m=+0.393786572,LastTimestamp:2026-07-02 23:10:08.398538288 +0000 UTC m=+0.393786572,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-84.ec2.internal,}" Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.412807 2570 factory.go:153] Registering CRI-O factory Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.412825 2570 factory.go:223] Registration of the crio container factory successfully Jul 02 23:10:08.858890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.412872 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.412880 2570 factory.go:55] Registering systemd factory Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.412886 2570 factory.go:223] Registration of the systemd container factory successfully Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.412902 2570 factory.go:103] Registering Raw factory Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.412914 2570 manager.go:1196] Started watching for new ooms in manager Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.415992 2570 manager.go:349] Starting recovery of all containers Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.416297 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-84.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.416606 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.427330 2570 manager.go:354] Recovery completed Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.427766 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xp8dl" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.432050 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.434091 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.434118 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.434132 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.434622 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.434624 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.434680 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.434697 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.435135 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xp8dl" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.435385 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.435420 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.435436 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.435804 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-84.ec2.internal.18be9bcd463a0880 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-84.ec2.internal,UID:ip-10-0-136-84.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-84.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-84.ec2.internal,},FirstTimestamp:2026-07-02 23:10:08.434104448 +0000 UTC m=+0.429352731,LastTimestamp:2026-07-02 23:10:08.434104448 +0000 UTC m=+0.429352731,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-84.ec2.internal,}" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.436066 2570 policy_none.go:49] "None policy: Start" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.436078 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.436089 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.436985 2570 policy_none.go:47] "Start" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.509949 2570 manager.go:337] "Starting Device Plugin manager" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.509996 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.510008 2570 server.go:87] "Starting device plugin registration server" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.510118 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.510429 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.510445 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.510537 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.510678 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.510688 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.511318 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.511371 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.584816 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.586037 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.586083 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 23:10:08.859986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.586111 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.586158 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.588961 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.611028 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.612039 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.612069 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.612080 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.612108 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.623910 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.623932 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-84.ec2.internal\": node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.644917 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.686973 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal"] Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.687087 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.692782 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.692817 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.692832 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.694087 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.694508 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.694558 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.695309 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.695330 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.695340 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.695583 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.695605 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.695618 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.697235 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.697262 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.697900 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientMemory" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.697922 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.697935 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeHasSufficientPID" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.712097 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b13cfcfee039b7d7f542ef70587b958d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"b13cfcfee039b7d7f542ef70587b958d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.712118 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b13cfcfee039b7d7f542ef70587b958d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"b13cfcfee039b7d7f542ef70587b958d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.712137 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/157508bb1f75054b68662588baddc867-config\") pod \"kube-apiserver-proxy-ip-10-0-136-84.ec2.internal\" (UID: \"157508bb1f75054b68662588baddc867\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.741974 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-84.ec2.internal\" not found" node="ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.861124 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.745167 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:08.862030 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.746633 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-84.ec2.internal\" not found" node="ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.862030 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.812778 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b13cfcfee039b7d7f542ef70587b958d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"b13cfcfee039b7d7f542ef70587b958d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.862030 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.812808 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/157508bb1f75054b68662588baddc867-config\") pod \"kube-apiserver-proxy-ip-10-0-136-84.ec2.internal\" (UID: \"157508bb1f75054b68662588baddc867\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.862030 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.812826 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b13cfcfee039b7d7f542ef70587b958d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"b13cfcfee039b7d7f542ef70587b958d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.862030 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.812869 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/157508bb1f75054b68662588baddc867-config\") pod \"kube-apiserver-proxy-ip-10-0-136-84.ec2.internal\" (UID: \"157508bb1f75054b68662588baddc867\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.862030 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.812876 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b13cfcfee039b7d7f542ef70587b958d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"b13cfcfee039b7d7f542ef70587b958d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.862030 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:08.812923 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b13cfcfee039b7d7f542ef70587b958d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal\" (UID: \"b13cfcfee039b7d7f542ef70587b958d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:08.862030 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.845915 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:08.970400 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:08.946765 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.058717 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.045313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:09.058717 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.047397 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.058717 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.050045 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" Jul 02 23:10:09.166061 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.147865 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.299003 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.248304 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.420398 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.344730 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 23:10:09.420398 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.344907 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 23:10:09.420398 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.349221 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.546115 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.437666 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 23:05:09 +0000 UTC" deadline="2028-02-09 15:31:49.503012207 +0000 UTC" Jul 02 23:10:09.546115 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.437707 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14080h21m40.065310345s" Jul 02 23:10:09.546115 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.449688 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.551113 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.550824 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.697813 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.578981 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:09.697813 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.651322 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.803628 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.735536 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:09.803628 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.752120 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.910122 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:09.830252 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13cfcfee039b7d7f542ef70587b958d.slice/crio-c27a4d4e4705ab563cf801f816fca1b91b9ce65b85bcb92886c9ecf960c76726 WatchSource:0}: Error finding container c27a4d4e4705ab563cf801f816fca1b91b9ce65b85bcb92886c9ecf960c76726: Status 404 returned error can't find the container with id c27a4d4e4705ab563cf801f816fca1b91b9ce65b85bcb92886c9ecf960c76726 Jul 02 23:10:09.910122 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:09.830705 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157508bb1f75054b68662588baddc867.slice/crio-8b854cd7a861744fba666af68220e42204b9c3dbc72be4aca7cc99f89572786c WatchSource:0}: Error finding container 8b854cd7a861744fba666af68220e42204b9c3dbc72be4aca7cc99f89572786c: Status 404 returned error can't find the container with id 8b854cd7a861744fba666af68220e42204b9c3dbc72be4aca7cc99f89572786c Jul 02 23:10:09.910122 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.838800 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:10:09.910122 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:09.852322 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-84.ec2.internal\" not found" Jul 02 23:10:09.910122 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.899783 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:09.911942 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.911284 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" Jul 02 23:10:10.033647 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.925156 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:10:10.033647 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.926596 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" Jul 02 23:10:10.033647 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:09.938878 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.389878 2570 apiserver.go:52] "Watching apiserver" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.398984 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.400134 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-rrq2p","openshift-multus/network-metrics-daemon-4spv8","kube-system/global-pull-secret-syncer-6zz8v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg","openshift-dns/node-resolver-mdw9r","openshift-network-diagnostics/network-check-target-zlls4","openshift-network-operator/iptables-alerter-n55jt","openshift-ovn-kubernetes/ovnkube-node-g6gs8","kube-system/konnectivity-agent-v9zn5","kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal","openshift-cluster-node-tuning-operator/tuned-xhm7n","openshift-image-registry/node-ca-fv8lb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal","openshift-multus/multus-additional-cni-plugins-6qnb6"] Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.404179 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.406748 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.407243 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4q6rp\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.407345 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.407488 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.407586 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.410100 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.410346 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.410383 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gnrxq\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.410564 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.410577 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.410625 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.411208 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.413619 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.414161 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g6d92\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.416063 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.416132 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.416192 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.416536 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.416774 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.417509 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.418516 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hmc9w\"" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.419677 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.421751 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysconfig\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.445070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.421780 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.421799 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.421830 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-kubernetes\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.421857 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysctl-d\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.421939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-tmp-dir\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.421982 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/baee3d99-fea8-4020-99a1-db02cfe2f3ae-host-slash\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422015 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysctl-conf\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422047 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-systemd\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422064 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1352ad10-5c46-4abe-ad7e-9c7d3572be46-dbus\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422083 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422105 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422136 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-socket-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422160 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f24nf\" (UniqueName: \"kubernetes.io/projected/baee3d99-fea8-4020-99a1-db02cfe2f3ae-kube-api-access-f24nf\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-sys\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422194 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1352ad10-5c46-4abe-ad7e-9c7d3572be46-kubelet-config\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422213 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-var-lib-kubelet\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.447654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422252 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpdmq\" (UniqueName: \"kubernetes.io/projected/acad8dcf-669c-435d-ae40-122403bad017-kube-api-access-kpdmq\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422284 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-device-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422307 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-sys-fs\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422337 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg8j6\" (UniqueName: \"kubernetes.io/projected/1a221851-7b76-4c44-845c-47ae871d19bb-kube-api-access-gg8j6\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422358 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-modprobe-d\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422380 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/acad8dcf-669c-435d-ae40-122403bad017-etc-tuned\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422400 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-run\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422416 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acad8dcf-669c-435d-ae40-122403bad017-tmp\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422435 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-registration-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422562 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-hosts-file\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422603 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlt5z\" (UniqueName: \"kubernetes.io/projected/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-kube-api-access-jlt5z\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422637 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/baee3d99-fea8-4020-99a1-db02cfe2f3ae-iptables-alerter-script\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422664 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-lib-modules\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.422703 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-host\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.423246 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8wphz\"" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.423476 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.429817 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.429990 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.447390 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.447638 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-87rng" Jul 02 23:10:10.516144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.449582 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.449673 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.450305 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.450318 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.450469 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.450707 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.450913 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nngrm\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.450942 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.451849 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-87rng" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.452363 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.454817 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.454984 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.455388 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.456285 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.456496 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.456534 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.456709 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wdfwz\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.456742 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.458717 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.459049 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.459060 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-44tqh\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.461517 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.461752 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.461766 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-97wff\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.461804 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 23:10:10.517818 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.512291 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523201 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-socket-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523243 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f24nf\" (UniqueName: \"kubernetes.io/projected/baee3d99-fea8-4020-99a1-db02cfe2f3ae-kube-api-access-f24nf\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523284 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-sys\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523349 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-sys\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg8j6\" (UniqueName: \"kubernetes.io/projected/1a221851-7b76-4c44-845c-47ae871d19bb-kube-api-access-gg8j6\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523399 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/55620e5e-72b6-4a11-afd9-f85e0171ba51-serviceca\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523435 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-os-release\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523446 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-socket-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523456 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpdmq\" (UniqueName: \"kubernetes.io/projected/acad8dcf-669c-435d-ae40-122403bad017-kube-api-access-kpdmq\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523640 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-device-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523668 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-run-netns\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523685 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovn-node-metrics-cert\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523705 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-k8s-cni-cncf-io\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523723 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-cni-bin\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523739 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-daemon-config\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523759 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-modprobe-d\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.526163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523765 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-device-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523782 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523818 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523894 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-cni-netd\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523893 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-modprobe-d\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523930 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-kubelet\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-etc-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.523988 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-kubelet\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524024 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-run\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acad8dcf-669c-435d-ae40-122403bad017-tmp\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524070 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlt5z\" (UniqueName: \"kubernetes.io/projected/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-kube-api-access-jlt5z\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524130 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-node-log\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524146 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-run\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524153 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovnkube-script-lib\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524190 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-cni-multus\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524217 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/baee3d99-fea8-4020-99a1-db02cfe2f3ae-iptables-alerter-script\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524240 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-host\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.527082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524283 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-system-cni-dir\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524343 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2qfg\" (UniqueName: \"kubernetes.io/projected/9c2e3cc3-5a74-4577-81ad-4439993bfb17-kube-api-access-j2qfg\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524396 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-system-cni-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524418 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-os-release\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524417 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524442 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-cni-binary-copy\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysctl-conf\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524732 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1352ad10-5c46-4abe-ad7e-9c7d3572be46-dbus\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524762 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-socket-dir-parent\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524791 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524887 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-etc-selinux\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.524958 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1352ad10-5c46-4abe-ad7e-9c7d3572be46-dbus\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525008 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-host\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525013 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysctl-conf\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525045 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.525099 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525126 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/baee3d99-fea8-4020-99a1-db02cfe2f3ae-iptables-alerter-script\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.527944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525139 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1352ad10-5c46-4abe-ad7e-9c7d3572be46-kubelet-config\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-sys-fs\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525207 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-sys-fs\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525220 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1352ad10-5c46-4abe-ad7e-9c7d3572be46-kubelet-config\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525242 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cnibin\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.525258 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret podName:1352ad10-5c46-4abe-ad7e-9c7d3572be46 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:11.025234252 +0000 UTC m=+3.020482546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret") pod "global-pull-secret-syncer-6zz8v" (UID: "1352ad10-5c46-4abe-ad7e-9c7d3572be46") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525282 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525335 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-var-lib-kubelet\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525395 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-var-lib-kubelet\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525435 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-var-lib-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525481 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-netns\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525510 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-multus-certs\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525537 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/acad8dcf-669c-435d-ae40-122403bad017-etc-tuned\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525579 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-cni-bin\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525602 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j97fm\" (UniqueName: \"kubernetes.io/projected/3dca5a94-9f5a-4269-b9b6-5394393fa89f-kube-api-access-j97fm\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525693 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-etc-kubernetes\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.528891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525748 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9mrk\" (UniqueName: \"kubernetes.io/projected/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-kube-api-access-j9mrk\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525778 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-systemd-units\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525823 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-registration-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525853 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-hosts-file\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525881 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f-agent-certs\") pod \"konnectivity-agent-v9zn5\" (UID: \"b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f\") " pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525908 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525949 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525969 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-hosts-file\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525975 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1a221851-7b76-4c44-845c-47ae871d19bb-registration-dir\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.525994 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-ovn\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526023 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-lib-modules\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526043 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4g4z\" (UniqueName: \"kubernetes.io/projected/55620e5e-72b6-4a11-afd9-f85e0171ba51-kube-api-access-w4g4z\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526062 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526091 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526123 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysconfig\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526143 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-lib-modules\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526151 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f-konnectivity-ca\") pod \"konnectivity-agent-v9zn5\" (UID: \"b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f\") " pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:10.529851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526204 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-env-overrides\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526237 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-cni-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526260 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-conf-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526277 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysconfig\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526288 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-kubernetes\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526316 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysctl-d\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-tmp-dir\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526347 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-kubernetes\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526452 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55620e5e-72b6-4a11-afd9-f85e0171ba51-host\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526447 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-sysctl-d\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526497 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-slash\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526528 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovnkube-config\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526554 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-hostroot\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526584 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/baee3d99-fea8-4020-99a1-db02cfe2f3ae-host-slash\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526613 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-systemd\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526635 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-tmp-dir\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526639 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cni-binary-copy\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.530701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526712 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/baee3d99-fea8-4020-99a1-db02cfe2f3ae-host-slash\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526771 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/acad8dcf-669c-435d-ae40-122403bad017-etc-systemd\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526777 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g422t\" (UniqueName: \"kubernetes.io/projected/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-kube-api-access-g422t\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526806 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-log-socket\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526832 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-run-ovn-kubernetes\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526862 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-cnibin\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.526887 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-systemd\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.529100 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/acad8dcf-669c-435d-ae40-122403bad017-tmp\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.531481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.529124 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/acad8dcf-669c-435d-ae40-122403bad017-etc-tuned\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.539275 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.536369 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg8j6\" (UniqueName: \"kubernetes.io/projected/1a221851-7b76-4c44-845c-47ae871d19bb-kube-api-access-gg8j6\") pod \"aws-ebs-csi-driver-node-fgmvg\" (UID: \"1a221851-7b76-4c44-845c-47ae871d19bb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.539275 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.536438 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlt5z\" (UniqueName: \"kubernetes.io/projected/5ef83259-01b2-4ae4-bdbc-af8081c9fa8a-kube-api-access-jlt5z\") pod \"node-resolver-mdw9r\" (UID: \"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a\") " pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.539705 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.539669 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f24nf\" (UniqueName: \"kubernetes.io/projected/baee3d99-fea8-4020-99a1-db02cfe2f3ae-kube-api-access-f24nf\") pod \"iptables-alerter-n55jt\" (UID: \"baee3d99-fea8-4020-99a1-db02cfe2f3ae\") " pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.540042 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.540018 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpdmq\" (UniqueName: \"kubernetes.io/projected/acad8dcf-669c-435d-ae40-122403bad017-kube-api-access-kpdmq\") pod \"tuned-xhm7n\" (UID: \"acad8dcf-669c-435d-ae40-122403bad017\") " pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.592025 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.591961 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" event={"ID":"157508bb1f75054b68662588baddc867","Type":"ContainerStarted","Data":"8b854cd7a861744fba666af68220e42204b9c3dbc72be4aca7cc99f89572786c"} Jul 02 23:10:10.593812 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.593777 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" event={"ID":"b13cfcfee039b7d7f542ef70587b958d","Type":"ContainerStarted","Data":"c27a4d4e4705ab563cf801f816fca1b91b9ce65b85bcb92886c9ecf960c76726"} Jul 02 23:10:10.628140 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628103 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55620e5e-72b6-4a11-afd9-f85e0171ba51-host\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.628313 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628147 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-slash\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.628313 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628176 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovnkube-config\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.628313 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628217 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-hostroot\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.628313 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628247 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cni-binary-copy\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.628313 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628251 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55620e5e-72b6-4a11-afd9-f85e0171ba51-host\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.628313 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628314 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-hostroot\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628251 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-slash\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628364 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g422t\" (UniqueName: \"kubernetes.io/projected/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-kube-api-access-g422t\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-log-socket\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628445 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-run-ovn-kubernetes\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628474 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-cnibin\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628500 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-systemd\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628533 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/55620e5e-72b6-4a11-afd9-f85e0171ba51-serviceca\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628558 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-os-release\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628586 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-run-netns\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628611 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovn-node-metrics-cert\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.628642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628637 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-k8s-cni-cncf-io\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628665 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-cni-bin\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628689 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-daemon-config\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628717 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628758 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628786 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-cni-netd\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628811 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-kubelet\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628837 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-etc-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-kubelet\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628890 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-node-log\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628897 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cni-binary-copy\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovnkube-script-lib\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628900 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovnkube-config\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628962 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-cni-multus\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628966 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-run-ovn-kubernetes\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.628994 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-system-cni-dir\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629024 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-cnibin\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.629286 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629025 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-run-netns\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629025 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2qfg\" (UniqueName: \"kubernetes.io/projected/9c2e3cc3-5a74-4577-81ad-4439993bfb17-kube-api-access-j2qfg\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629066 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-kubelet\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629069 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-os-release\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629080 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-system-cni-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629078 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-systemd\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629075 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-etc-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629131 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-cni-bin\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629134 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-system-cni-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629162 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-os-release\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629198 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629219 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-cni-binary-copy\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629257 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-socket-dir-parent\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629305 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cnibin\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629333 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-var-lib-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629378 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-var-lib-openvswitch\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629387 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-system-cni-dir\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.630203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629403 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629435 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-log-socket\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629471 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-netns\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629498 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-multus-certs\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629527 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-cni-bin\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j97fm\" (UniqueName: \"kubernetes.io/projected/3dca5a94-9f5a-4269-b9b6-5394393fa89f-kube-api-access-j97fm\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629581 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-etc-kubernetes\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9mrk\" (UniqueName: \"kubernetes.io/projected/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-kube-api-access-j9mrk\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629631 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-systemd-units\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629658 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f-agent-certs\") pod \"konnectivity-agent-v9zn5\" (UID: \"b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f\") " pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629685 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629713 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629739 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-ovn\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629738 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629787 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovnkube-script-lib\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629794 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-daemon-config\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-netns\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629788 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-cni-netd\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.631095 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629812 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4g4z\" (UniqueName: \"kubernetes.io/projected/55620e5e-72b6-4a11-afd9-f85e0171ba51-kube-api-access-w4g4z\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629846 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-cni-bin\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629846 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/55620e5e-72b6-4a11-afd9-f85e0171ba51-serviceca\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629897 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-k8s-cni-cncf-io\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629902 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-socket-dir-parent\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629946 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-var-lib-cni-multus\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629987 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-cnibin\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630000 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-os-release\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630046 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-host-run-multus-certs\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.629847 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630077 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630096 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f-konnectivity-ca\") pod \"konnectivity-agent-v9zn5\" (UID: \"b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f\") " pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630115 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-env-overrides\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630131 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-cni-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630147 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-conf-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630166 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9c2e3cc3-5a74-4577-81ad-4439993bfb17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630195 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-conf-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630219 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630219 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-systemd-units\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.630253 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630307 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-node-log\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630306 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-cni-binary-copy\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630343 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-host-kubelet\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.630352 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs podName:84f0d6e1-770c-4014-a4dc-7d27e5a7b1db nodeName:}" failed. No retries permitted until 2026-07-02 23:10:11.130332021 +0000 UTC m=+3.125580301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs") pod "network-metrics-daemon-4spv8" (UID: "84f0d6e1-770c-4014-a4dc-7d27e5a7b1db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630384 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-etc-kubernetes\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630424 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3dca5a94-9f5a-4269-b9b6-5394393fa89f-run-ovn\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630494 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-multus-cni-dir\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630672 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3dca5a94-9f5a-4269-b9b6-5394393fa89f-env-overrides\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.632854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.630868 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f-konnectivity-ca\") pod \"konnectivity-agent-v9zn5\" (UID: \"b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f\") " pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:10.633961 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.633937 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f-agent-certs\") pod \"konnectivity-agent-v9zn5\" (UID: \"b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f\") " pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:10.634071 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.633962 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3dca5a94-9f5a-4269-b9b6-5394393fa89f-ovn-node-metrics-cert\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.639043 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.639012 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:10.639043 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.639036 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:10.639043 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.639046 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qzlcv for pod openshift-network-diagnostics/network-check-target-zlls4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:10.639301 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:10.639099 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv podName:27b91be6-943e-4daf-b4af-aa7947d2993e nodeName:}" failed. No retries permitted until 2026-07-02 23:10:11.139085594 +0000 UTC m=+3.134333864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qzlcv" (UniqueName: "kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv") pod "network-check-target-zlls4" (UID: "27b91be6-943e-4daf-b4af-aa7947d2993e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:10.641143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.641117 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2qfg\" (UniqueName: \"kubernetes.io/projected/9c2e3cc3-5a74-4577-81ad-4439993bfb17-kube-api-access-j2qfg\") pod \"multus-additional-cni-plugins-6qnb6\" (UID: \"9c2e3cc3-5a74-4577-81ad-4439993bfb17\") " pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.641729 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.641680 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4g4z\" (UniqueName: \"kubernetes.io/projected/55620e5e-72b6-4a11-afd9-f85e0171ba51-kube-api-access-w4g4z\") pod \"node-ca-fv8lb\" (UID: \"55620e5e-72b6-4a11-afd9-f85e0171ba51\") " pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.641729 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.641699 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g422t\" (UniqueName: \"kubernetes.io/projected/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-kube-api-access-g422t\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:10.642437 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.642371 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9mrk\" (UniqueName: \"kubernetes.io/projected/61673e8e-ad7f-482f-8b58-2dfdb5d81ddc-kube-api-access-j9mrk\") pod \"multus-rrq2p\" (UID: \"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc\") " pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.642923 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.642899 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j97fm\" (UniqueName: \"kubernetes.io/projected/3dca5a94-9f5a-4269-b9b6-5394393fa89f-kube-api-access-j97fm\") pod \"ovnkube-node-g6gs8\" (UID: \"3dca5a94-9f5a-4269-b9b6-5394393fa89f\") " pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.718509 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.718416 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-n55jt" Jul 02 23:10:10.730024 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:10.729979 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaee3d99_fea8_4020_99a1_db02cfe2f3ae.slice/crio-616d6ade5245479050be26d4cca19c2e75c2c2d64cd145543a9956290179fad7 WatchSource:0}: Error finding container 616d6ade5245479050be26d4cca19c2e75c2c2d64cd145543a9956290179fad7: Status 404 returned error can't find the container with id 616d6ade5245479050be26d4cca19c2e75c2c2d64cd145543a9956290179fad7 Jul 02 23:10:10.747497 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.747464 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" Jul 02 23:10:10.751908 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.751880 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" Jul 02 23:10:10.758328 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.758280 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mdw9r" Jul 02 23:10:10.759118 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:10.758814 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacad8dcf_669c_435d_ae40_122403bad017.slice/crio-a222c17e95129fcf5ea032ee7c52a0cf8d54f83488cfa2339b04cfba1eb29242 WatchSource:0}: Error finding container a222c17e95129fcf5ea032ee7c52a0cf8d54f83488cfa2339b04cfba1eb29242: Status 404 returned error can't find the container with id a222c17e95129fcf5ea032ee7c52a0cf8d54f83488cfa2339b04cfba1eb29242 Jul 02 23:10:10.765814 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.765791 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:10.772005 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.771978 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" Jul 02 23:10:10.774966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.774945 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 23:10:10.777672 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.777650 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:10.777928 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:10.777903 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef83259_01b2_4ae4_bdbc_af8081c9fa8a.slice/crio-5e5f47ad2d47808e83162f505c9f87f7ceaeb827ce9c633c9d7b83a6ffe0156a WatchSource:0}: Error finding container 5e5f47ad2d47808e83162f505c9f87f7ceaeb827ce9c633c9d7b83a6ffe0156a: Status 404 returned error can't find the container with id 5e5f47ad2d47808e83162f505c9f87f7ceaeb827ce9c633c9d7b83a6ffe0156a Jul 02 23:10:10.779767 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:10.779737 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a221851_7b76_4c44_845c_47ae871d19bb.slice/crio-939db2e34c81e3faa8c86a2f81574455cbead5d68c098c390393527a1fe7248a WatchSource:0}: Error finding container 939db2e34c81e3faa8c86a2f81574455cbead5d68c098c390393527a1fe7248a: Status 404 returned error can't find the container with id 939db2e34c81e3faa8c86a2f81574455cbead5d68c098c390393527a1fe7248a Jul 02 23:10:10.788132 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.787982 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rrq2p" Jul 02 23:10:10.788437 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:10.788414 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fv8lb" Jul 02 23:10:10.795060 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:10.795031 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2e3cc3_5a74_4577_81ad_4439993bfb17.slice/crio-247427d1a6fb7f42a62e8864082791a0b50d1504ecd33e2f3587e71223212440 WatchSource:0}: Error finding container 247427d1a6fb7f42a62e8864082791a0b50d1504ecd33e2f3587e71223212440: Status 404 returned error can't find the container with id 247427d1a6fb7f42a62e8864082791a0b50d1504ecd33e2f3587e71223212440 Jul 02 23:10:10.811830 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:10.811403 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55620e5e_72b6_4a11_afd9_f85e0171ba51.slice/crio-39bc3b23bb55aa9cb21290a018d33e5039cc9584bcbf8a8b5601383f2c8b8b9a WatchSource:0}: Error finding container 39bc3b23bb55aa9cb21290a018d33e5039cc9584bcbf8a8b5601383f2c8b8b9a: Status 404 returned error can't find the container with id 39bc3b23bb55aa9cb21290a018d33e5039cc9584bcbf8a8b5601383f2c8b8b9a Jul 02 23:10:11.034780 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.034682 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:11.034942 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:11.034862 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:11.034989 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:11.034951 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret podName:1352ad10-5c46-4abe-ad7e-9c7d3572be46 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:12.03492785 +0000 UTC m=+4.030176127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret") pod "global-pull-secret-syncer-6zz8v" (UID: "1352ad10-5c46-4abe-ad7e-9c7d3572be46") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:11.135927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.135880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:11.136153 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:11.136129 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:11.136246 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:11.136234 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs podName:84f0d6e1-770c-4014-a4dc-7d27e5a7b1db nodeName:}" failed. No retries permitted until 2026-07-02 23:10:12.136195339 +0000 UTC m=+4.131443625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs") pod "network-metrics-daemon-4spv8" (UID: "84f0d6e1-770c-4014-a4dc-7d27e5a7b1db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:11.236962 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.236921 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:11.237162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:11.237127 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:11.237162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:11.237151 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:11.237162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:11.237162 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qzlcv for pod openshift-network-diagnostics/network-check-target-zlls4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:11.237358 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:11.237241 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv podName:27b91be6-943e-4daf-b4af-aa7947d2993e nodeName:}" failed. No retries permitted until 2026-07-02 23:10:12.237220905 +0000 UTC m=+4.232469181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzlcv" (UniqueName: "kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv") pod "network-check-target-zlls4" (UID: "27b91be6-943e-4daf-b4af-aa7947d2993e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:11.455634 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.455570 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:05:10 +0000 UTC" deadline="2028-03-27 04:09:17.327465782 +0000 UTC" Jul 02 23:10:11.455634 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.455623 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15196h59m5.871845834s" Jul 02 23:10:11.597908 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.597794 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fv8lb" event={"ID":"55620e5e-72b6-4a11-afd9-f85e0171ba51","Type":"ContainerStarted","Data":"39bc3b23bb55aa9cb21290a018d33e5039cc9584bcbf8a8b5601383f2c8b8b9a"} Jul 02 23:10:11.599148 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.599115 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"ce03c30a6d6d356984f36e0182a759476de3714bab98ed6a41ffbffe9047f84f"} Jul 02 23:10:11.600950 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.600922 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerStarted","Data":"247427d1a6fb7f42a62e8864082791a0b50d1504ecd33e2f3587e71223212440"} Jul 02 23:10:11.602400 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.602339 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" event={"ID":"1a221851-7b76-4c44-845c-47ae871d19bb","Type":"ContainerStarted","Data":"939db2e34c81e3faa8c86a2f81574455cbead5d68c098c390393527a1fe7248a"} Jul 02 23:10:11.604162 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.604135 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" event={"ID":"acad8dcf-669c-435d-ae40-122403bad017","Type":"ContainerStarted","Data":"a222c17e95129fcf5ea032ee7c52a0cf8d54f83488cfa2339b04cfba1eb29242"} Jul 02 23:10:11.605692 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.605661 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n55jt" event={"ID":"baee3d99-fea8-4020-99a1-db02cfe2f3ae","Type":"ContainerStarted","Data":"616d6ade5245479050be26d4cca19c2e75c2c2d64cd145543a9956290179fad7"} Jul 02 23:10:11.607064 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.607008 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rrq2p" event={"ID":"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc","Type":"ContainerStarted","Data":"3bff2b612f9aa5b8e609791d81136ae119bb1e36d0794c8587c38694953c9d89"} Jul 02 23:10:11.609350 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.608817 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v9zn5" event={"ID":"b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f","Type":"ContainerStarted","Data":"1a43583e27c20622e987d9e01ab5c6e26f1b8fa587f8dd6b5f0da0286e06c707"} Jul 02 23:10:11.611118 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:11.611059 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mdw9r" event={"ID":"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a","Type":"ContainerStarted","Data":"5e5f47ad2d47808e83162f505c9f87f7ceaeb827ce9c633c9d7b83a6ffe0156a"} Jul 02 23:10:12.043622 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:12.043571 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:12.043826 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.043805 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:12.043891 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.043882 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret podName:1352ad10-5c46-4abe-ad7e-9c7d3572be46 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:14.043862751 +0000 UTC m=+6.039111024 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret") pod "global-pull-secret-syncer-6zz8v" (UID: "1352ad10-5c46-4abe-ad7e-9c7d3572be46") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:12.145135 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:12.144507 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:12.145135 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.144686 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:12.145135 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.144756 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs podName:84f0d6e1-770c-4014-a4dc-7d27e5a7b1db nodeName:}" failed. No retries permitted until 2026-07-02 23:10:14.14473543 +0000 UTC m=+6.139983703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs") pod "network-metrics-daemon-4spv8" (UID: "84f0d6e1-770c-4014-a4dc-7d27e5a7b1db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:12.245631 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:12.245588 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:12.245830 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.245790 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:12.245830 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.245811 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:12.245830 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.245825 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qzlcv for pod openshift-network-diagnostics/network-check-target-zlls4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:12.245993 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.245887 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv podName:27b91be6-943e-4daf-b4af-aa7947d2993e nodeName:}" failed. No retries permitted until 2026-07-02 23:10:14.245868606 +0000 UTC m=+6.241116880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzlcv" (UniqueName: "kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv") pod "network-check-target-zlls4" (UID: "27b91be6-943e-4daf-b4af-aa7947d2993e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:12.456476 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:12.456424 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:05:10 +0000 UTC" deadline="2027-12-15 18:36:27.994638576 +0000 UTC" Jul 02 23:10:12.456476 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:12.456473 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12739h26m15.53816996s" Jul 02 23:10:12.586547 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:12.586483 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:12.586725 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.586649 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:12.587134 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:12.587109 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:12.587248 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.587225 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:12.587358 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:12.587313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:12.587426 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:12.587394 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:13.638375 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:13.635658 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" event={"ID":"157508bb1f75054b68662588baddc867","Type":"ContainerStarted","Data":"684eb2596394275d305625ccb049fdb3a7b7f95a1b3f41700d59cd956209da3d"} Jul 02 23:10:13.652748 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:13.652710 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" event={"ID":"b13cfcfee039b7d7f542ef70587b958d","Type":"ContainerStarted","Data":"56c4fca7094ec8cd2b574f844bf772360046841a792e6ca72a04a1ce5d8d6075"} Jul 02 23:10:13.657682 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:13.657626 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-84.ec2.internal" podStartSLOduration=3.657607116 podStartE2EDuration="3.657607116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:13.657223976 +0000 UTC m=+5.652472263" watchObservedRunningTime="2026-07-02 23:10:13.657607116 +0000 UTC m=+5.652855412" Jul 02 23:10:14.062484 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:14.062438 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:14.062668 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.062623 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:14.062765 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.062709 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret podName:1352ad10-5c46-4abe-ad7e-9c7d3572be46 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:18.062687201 +0000 UTC m=+10.057935472 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret") pod "global-pull-secret-syncer-6zz8v" (UID: "1352ad10-5c46-4abe-ad7e-9c7d3572be46") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:14.163135 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:14.163094 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:14.163359 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.163277 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:14.163359 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.163348 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs podName:84f0d6e1-770c-4014-a4dc-7d27e5a7b1db nodeName:}" failed. No retries permitted until 2026-07-02 23:10:18.163327972 +0000 UTC m=+10.158576247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs") pod "network-metrics-daemon-4spv8" (UID: "84f0d6e1-770c-4014-a4dc-7d27e5a7b1db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:14.264400 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:14.264339 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:14.264609 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.264546 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:14.264609 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.264571 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:14.264609 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.264587 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qzlcv for pod openshift-network-diagnostics/network-check-target-zlls4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:14.264772 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.264647 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv podName:27b91be6-943e-4daf-b4af-aa7947d2993e nodeName:}" failed. No retries permitted until 2026-07-02 23:10:18.264628446 +0000 UTC m=+10.259876732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzlcv" (UniqueName: "kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv") pod "network-check-target-zlls4" (UID: "27b91be6-943e-4daf-b4af-aa7947d2993e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:14.586591 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:14.586552 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:14.586795 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.586700 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:14.587169 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:14.587141 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:14.587303 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.587281 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:14.587384 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:14.587356 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:14.590388 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:14.587441 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:14.660598 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:14.660019 2570 generic.go:356] "Generic (PLEG): container finished" podID="b13cfcfee039b7d7f542ef70587b958d" containerID="56c4fca7094ec8cd2b574f844bf772360046841a792e6ca72a04a1ce5d8d6075" exitCode=0 Jul 02 23:10:14.660598 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:14.660335 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" event={"ID":"b13cfcfee039b7d7f542ef70587b958d","Type":"ContainerDied","Data":"56c4fca7094ec8cd2b574f844bf772360046841a792e6ca72a04a1ce5d8d6075"} Jul 02 23:10:16.590480 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:16.590446 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:16.590480 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:16.590479 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:16.590972 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:16.590446 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:16.590972 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:16.590590 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:16.590972 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:16.590678 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:16.590972 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:16.590759 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:18.100610 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:18.100574 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:18.101106 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.100761 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:18.101106 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.100847 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret podName:1352ad10-5c46-4abe-ad7e-9c7d3572be46 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:26.100824481 +0000 UTC m=+18.096072770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret") pod "global-pull-secret-syncer-6zz8v" (UID: "1352ad10-5c46-4abe-ad7e-9c7d3572be46") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:18.201159 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:18.201120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:18.201337 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.201291 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:18.201413 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.201356 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs podName:84f0d6e1-770c-4014-a4dc-7d27e5a7b1db nodeName:}" failed. No retries permitted until 2026-07-02 23:10:26.201336221 +0000 UTC m=+18.196584507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs") pod "network-metrics-daemon-4spv8" (UID: "84f0d6e1-770c-4014-a4dc-7d27e5a7b1db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:18.302634 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:18.302434 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:18.302802 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.302659 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:18.302802 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.302680 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:18.302802 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.302692 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qzlcv for pod openshift-network-diagnostics/network-check-target-zlls4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:18.302802 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.302755 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv podName:27b91be6-943e-4daf-b4af-aa7947d2993e nodeName:}" failed. No retries permitted until 2026-07-02 23:10:26.302736208 +0000 UTC m=+18.297984501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzlcv" (UniqueName: "kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv") pod "network-check-target-zlls4" (UID: "27b91be6-943e-4daf-b4af-aa7947d2993e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:18.588171 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:18.588130 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:18.588362 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.588274 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:18.588964 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:18.588663 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:18.588964 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.588755 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:18.588964 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:18.588814 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:18.588964 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:18.588926 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:20.590729 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:20.590252 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:20.590729 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:20.590272 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:20.590729 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:20.590285 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:20.590729 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:20.590378 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:20.590729 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:20.590447 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:20.590729 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:20.590520 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:22.587273 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:22.587233 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:22.587740 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:22.587243 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:22.587740 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:22.587370 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:22.587740 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:22.587460 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:22.587740 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:22.587243 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:22.587740 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:22.587568 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:24.590330 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:24.590302 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:24.590330 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:24.590315 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:24.590775 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:24.590315 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:24.590775 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:24.590417 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:24.590775 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:24.590492 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:24.590775 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:24.590561 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:26.155360 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:26.155319 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:26.155806 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.155496 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:26.155806 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.155588 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret podName:1352ad10-5c46-4abe-ad7e-9c7d3572be46 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:42.15556451 +0000 UTC m=+34.150812793 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret") pod "global-pull-secret-syncer-6zz8v" (UID: "1352ad10-5c46-4abe-ad7e-9c7d3572be46") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:26.256124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:26.256092 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:26.256294 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.256261 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:26.256339 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.256326 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs podName:84f0d6e1-770c-4014-a4dc-7d27e5a7b1db nodeName:}" failed. No retries permitted until 2026-07-02 23:10:42.256312115 +0000 UTC m=+34.251560383 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs") pod "network-metrics-daemon-4spv8" (UID: "84f0d6e1-770c-4014-a4dc-7d27e5a7b1db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:26.356732 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:26.356698 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:26.356892 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.356842 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:26.356892 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.356859 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:26.356892 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.356867 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qzlcv for pod openshift-network-diagnostics/network-check-target-zlls4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:26.357004 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.356918 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv podName:27b91be6-943e-4daf-b4af-aa7947d2993e nodeName:}" failed. No retries permitted until 2026-07-02 23:10:42.356905309 +0000 UTC m=+34.352153577 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzlcv" (UniqueName: "kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv") pod "network-check-target-zlls4" (UID: "27b91be6-943e-4daf-b4af-aa7947d2993e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:26.586355 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:26.586326 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:26.586530 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:26.586359 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:26.586530 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:26.586326 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:26.586530 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.586447 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:26.586530 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.586505 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:26.586740 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:26.586595 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:28.587560 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:28.587521 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:28.587560 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:28.587547 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:28.588121 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:28.587643 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:28.588121 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:28.587677 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:28.588121 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:28.587721 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:28.588121 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:28.587800 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:30.588016 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.586908 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:30.588016 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:30.587393 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:30.588016 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.587482 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:30.588016 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:30.587567 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:30.588016 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.587629 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:30.588016 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:30.587708 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:30.691510 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.691468 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" event={"ID":"b13cfcfee039b7d7f542ef70587b958d","Type":"ContainerStarted","Data":"17cd4240744831531b18f6b56617ec660e24ef12d356dc534841ae5d96007c59"} Jul 02 23:10:30.692981 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.692950 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fv8lb" event={"ID":"55620e5e-72b6-4a11-afd9-f85e0171ba51","Type":"ContainerStarted","Data":"d2759f33b49b1d0b69374843d6e917d74fdc1da587e7cb3d9ab36757d041e190"} Jul 02 23:10:30.694521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.694497 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerStarted","Data":"7079c344a54f2fc935c4fca657764623e7a3ce2bca737120203805bd6646d9ab"} Jul 02 23:10:30.696231 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.696208 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" event={"ID":"1a221851-7b76-4c44-845c-47ae871d19bb","Type":"ContainerStarted","Data":"f503f70e48d39fba9c880983f71ed0c959b64fce07b0231f149a7814becfe82a"} Jul 02 23:10:30.697967 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.697939 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" event={"ID":"acad8dcf-669c-435d-ae40-122403bad017","Type":"ContainerStarted","Data":"020982937ab7b7b725f5581849b1784e1275551e2c6c7c3b2405ebd588f23a8c"} Jul 02 23:10:30.699808 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.699758 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rrq2p" event={"ID":"61673e8e-ad7f-482f-8b58-2dfdb5d81ddc","Type":"ContainerStarted","Data":"c8bd961bd9795895f37480fd66cc7aea01b9b0238bbd15e829e03230da3a0144"} Jul 02 23:10:30.701494 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.701472 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v9zn5" event={"ID":"b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f","Type":"ContainerStarted","Data":"e1fc7ce97f1bc0057c13a87de55e9a567257568a7679914eaefac9f253e9f1ae"} Jul 02 23:10:30.703340 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.703314 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mdw9r" event={"ID":"5ef83259-01b2-4ae4-bdbc-af8081c9fa8a","Type":"ContainerStarted","Data":"bb31e47c45aa2ac0e4ca0d105fbc0a10c241fc7b728f3154319ac647754a6d82"} Jul 02 23:10:30.714591 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.714398 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-84.ec2.internal" podStartSLOduration=20.714383316 podStartE2EDuration="20.714383316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:30.714141973 +0000 UTC m=+22.709390267" watchObservedRunningTime="2026-07-02 23:10:30.714383316 +0000 UTC m=+22.709631606" Jul 02 23:10:30.735624 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.735569 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xhm7n" podStartSLOduration=2.105658941 podStartE2EDuration="21.73554767s" totalImagesPullingTime="19.629888729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:30.735130422 +0000 UTC m=+22.730378715" watchObservedRunningTime="2026-07-02 23:10:30.73554767 +0000 UTC m=+22.730795962" Jul 02 23:10:30.780540 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.780488 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rrq2p" podStartSLOduration=2.162492765 podStartE2EDuration="21.780474772s" totalImagesPullingTime="19.617982007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:30.780259863 +0000 UTC m=+22.775508174" watchObservedRunningTime="2026-07-02 23:10:30.780474772 +0000 UTC m=+22.775723063" Jul 02 23:10:30.795072 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.795025 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mdw9r" podStartSLOduration=6.304505056 podStartE2EDuration="21.795009796s" totalImagesPullingTime="15.49050474s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:30.794677846 +0000 UTC m=+22.789926139" watchObservedRunningTime="2026-07-02 23:10:30.795009796 +0000 UTC m=+22.790258086" Jul 02 23:10:30.813579 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.813523 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v9zn5" podStartSLOduration=2.215695837 podStartE2EDuration="21.813501269s" totalImagesPullingTime="19.597805432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:30.813163633 +0000 UTC m=+22.808411921" watchObservedRunningTime="2026-07-02 23:10:30.813501269 +0000 UTC m=+22.808749562" Jul 02 23:10:30.829373 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:30.829328 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fv8lb" podStartSLOduration=2.252091961 podStartE2EDuration="21.829311976s" totalImagesPullingTime="19.577220015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:30.829165585 +0000 UTC m=+22.824413875" watchObservedRunningTime="2026-07-02 23:10:30.829311976 +0000 UTC m=+22.824560266" Jul 02 23:10:31.659911 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.659881 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:31.660652 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.660472 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:31.707879 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.707855 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:10:31.708223 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.708178 2570 generic.go:356] "Generic (PLEG): container finished" podID="3dca5a94-9f5a-4269-b9b6-5394393fa89f" containerID="7c6b9218518e92ebc4338388f8035cb0bd7404050cde14bb32a2c6f0bbc8230c" exitCode=1 Jul 02 23:10:31.708336 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.708223 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"b2f77e6882bdfb5409add098526a9809cb2f1afbee6927700c723ca49757e01a"} Jul 02 23:10:31.708336 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.708295 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"0cc1682360bd7f38049c4f8e64bc827ef4c2e2712349b9d75ec97cc71c2e9c22"} Jul 02 23:10:31.708336 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.708329 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"0a6167cf0a1f19e354548cdb26b178094a86878fe2979080df7c899355084b34"} Jul 02 23:10:31.708449 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.708366 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"75d5f1ba7c4555566f3b7ccb1670e35c830c826ed17db6c40ed41749b5e4ece2"} Jul 02 23:10:31.708449 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.708407 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerDied","Data":"7c6b9218518e92ebc4338388f8035cb0bd7404050cde14bb32a2c6f0bbc8230c"} Jul 02 23:10:31.708449 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.708441 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"63a5f5efb33a1ebae7c9ae99e14b8a365560b66e09320f9ebaa28a622559d629"} Jul 02 23:10:31.709585 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.709564 2570 generic.go:356] "Generic (PLEG): container finished" podID="9c2e3cc3-5a74-4577-81ad-4439993bfb17" containerID="7079c344a54f2fc935c4fca657764623e7a3ce2bca737120203805bd6646d9ab" exitCode=0 Jul 02 23:10:31.709686 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.709644 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerDied","Data":"7079c344a54f2fc935c4fca657764623e7a3ce2bca737120203805bd6646d9ab"} Jul 02 23:10:31.710989 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.710967 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-n55jt" event={"ID":"baee3d99-fea8-4020-99a1-db02cfe2f3ae","Type":"ContainerStarted","Data":"e189002f77956aa1cbff97095d5593db9019e8db67aee25292457db9ef589e31"} Jul 02 23:10:31.711877 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.711859 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:31.712049 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.711993 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v9zn5" Jul 02 23:10:31.770080 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.770031 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-n55jt" podStartSLOduration=7.225778232 podStartE2EDuration="22.770014592s" totalImagesPullingTime="15.54423636s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:31.752478919 +0000 UTC m=+23.747727224" watchObservedRunningTime="2026-07-02 23:10:31.770014592 +0000 UTC m=+23.765262882" Jul 02 23:10:31.961354 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:31.961330 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 23:10:32.526049 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:32.525929 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T23:10:31.961348968Z","UUID":"421bd2af-7f88-4146-97b4-cc567e835575","Handler":null,"Name":"","Endpoint":""} Jul 02 23:10:32.527842 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:32.527822 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 23:10:32.527967 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:32.527863 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 23:10:32.587205 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:32.587084 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:32.587404 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:32.587214 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:32.587404 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:32.587224 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:32.587404 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:32.587293 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:32.587404 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:32.587081 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:32.587404 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:32.587380 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:32.716754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:32.716702 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" event={"ID":"1a221851-7b76-4c44-845c-47ae871d19bb","Type":"ContainerStarted","Data":"1892a4c6bd6ec0b8d3c2eff728d4b9b455a731de1fa51e97a73cdf8d61cd249f"} Jul 02 23:10:33.723025 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:33.722780 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:10:33.723606 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:33.723415 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"d22a2ec7a95b99c2242cb9142fe68feacdc644aaebe1b7c325c91e962b5d7202"} Jul 02 23:10:33.725583 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:33.725556 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" event={"ID":"1a221851-7b76-4c44-845c-47ae871d19bb","Type":"ContainerStarted","Data":"c436d140eb54a2aa4f2005ab3424c8384121f8e4daafd6bd8eb62a471feb5f35"} Jul 02 23:10:33.743336 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:33.743260 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fgmvg" podStartSLOduration=2.809101621 podStartE2EDuration="24.743242036s" totalImagesPullingTime="21.934140415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:33.74297678 +0000 UTC m=+25.738225095" watchObservedRunningTime="2026-07-02 23:10:33.743242036 +0000 UTC m=+25.738490327" Jul 02 23:10:34.587252 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:34.587211 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:34.587252 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:34.587244 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:34.587504 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:34.587217 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:34.587504 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:34.587329 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:34.587504 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:34.587392 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:34.587504 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:34.587459 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:36.587265 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.587076 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:36.587848 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.587079 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:36.587848 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:36.587361 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:36.587848 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.587079 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:36.587848 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:36.587413 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:36.587848 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:36.587512 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:36.735558 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.735534 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:10:36.735872 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.735852 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"ff37a9babe3795ed620c23493a7cb51646445116289b56c84a5ca66c1af57d2b"} Jul 02 23:10:36.736164 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.736144 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:36.736290 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.736231 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:36.736290 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.736283 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:36.736507 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.736484 2570 scope.go:117] "RemoveContainer" containerID="7c6b9218518e92ebc4338388f8035cb0bd7404050cde14bb32a2c6f0bbc8230c" Jul 02 23:10:36.738036 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.738013 2570 generic.go:356] "Generic (PLEG): container finished" podID="9c2e3cc3-5a74-4577-81ad-4439993bfb17" containerID="17b20f9e5e12f160d83ef5510e4976c5f902e2f13915e891d597c19953b2e43a" exitCode=0 Jul 02 23:10:36.738146 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.738050 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerDied","Data":"17b20f9e5e12f160d83ef5510e4976c5f902e2f13915e891d597c19953b2e43a"} Jul 02 23:10:36.753169 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.753142 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:36.753387 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:36.753371 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:10:37.744572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:37.744545 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:10:37.745111 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:37.744857 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" event={"ID":"3dca5a94-9f5a-4269-b9b6-5394393fa89f","Type":"ContainerStarted","Data":"fd44696e79317186f7469fc10a00d3c1cd3dc3a7754955691e15c23c3f3fb531"} Jul 02 23:10:37.773136 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:37.773082 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" podStartSLOduration=8.926408262 podStartE2EDuration="28.773067116s" totalImagesPullingTime="19.846658854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:37.771378377 +0000 UTC m=+29.766626681" watchObservedRunningTime="2026-07-02 23:10:37.773067116 +0000 UTC m=+29.768315408" Jul 02 23:10:38.251600 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:38.251559 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6zz8v"] Jul 02 23:10:38.251756 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:38.251704 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:38.251855 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:38.251823 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:38.254937 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:38.254851 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zlls4"] Jul 02 23:10:38.255083 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:38.254975 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:38.255143 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:38.255082 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:38.255525 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:38.255504 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4spv8"] Jul 02 23:10:38.255645 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:38.255613 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:38.255733 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:38.255715 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:38.748948 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:38.748752 2570 generic.go:356] "Generic (PLEG): container finished" podID="9c2e3cc3-5a74-4577-81ad-4439993bfb17" containerID="4f60460cd196f28d6e580a7f5058bd8d55c36b6ce5f5e47069347777c91de6ac" exitCode=0 Jul 02 23:10:38.749383 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:38.748839 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerDied","Data":"4f60460cd196f28d6e580a7f5058bd8d55c36b6ce5f5e47069347777c91de6ac"} Jul 02 23:10:39.586464 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:39.586431 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:39.586658 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:39.586435 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:39.586658 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:39.586561 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:39.586658 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:39.586557 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:39.586809 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:39.586678 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:39.586809 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:39.586777 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:40.755721 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:40.755684 2570 generic.go:356] "Generic (PLEG): container finished" podID="9c2e3cc3-5a74-4577-81ad-4439993bfb17" containerID="cc5369eaab0ca482ac7ff4bfe3b99b1a2c5e5be24028d7db34d887d4b1bc0659" exitCode=0 Jul 02 23:10:40.756381 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:40.755728 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerDied","Data":"cc5369eaab0ca482ac7ff4bfe3b99b1a2c5e5be24028d7db34d887d4b1bc0659"} Jul 02 23:10:41.586905 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:41.586871 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:41.587087 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:41.586872 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:41.587087 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:41.586975 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:41.587087 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:41.587058 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:41.587087 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:41.586880 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:41.587241 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:41.587136 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:42.184194 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:42.184141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:42.184654 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:42.184303 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:42.184654 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:42.184378 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret podName:1352ad10-5c46-4abe-ad7e-9c7d3572be46 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:14.184362425 +0000 UTC m=+66.179610695 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret") pod "global-pull-secret-syncer-6zz8v" (UID: "1352ad10-5c46-4abe-ad7e-9c7d3572be46") : object "kube-system"/"original-pull-secret" not registered Jul 02 23:10:42.284850 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:42.284813 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:42.285021 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:42.284985 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:42.285084 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:42.285067 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs podName:84f0d6e1-770c-4014-a4dc-7d27e5a7b1db nodeName:}" failed. No retries permitted until 2026-07-02 23:11:14.285047184 +0000 UTC m=+66.280295458 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs") pod "network-metrics-daemon-4spv8" (UID: "84f0d6e1-770c-4014-a4dc-7d27e5a7b1db") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 23:10:42.385443 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:42.385407 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:42.385639 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:42.385599 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 23:10:42.385639 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:42.385628 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 23:10:42.385639 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:42.385642 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qzlcv for pod openshift-network-diagnostics/network-check-target-zlls4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:42.385800 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:42.385711 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv podName:27b91be6-943e-4daf-b4af-aa7947d2993e nodeName:}" failed. No retries permitted until 2026-07-02 23:11:14.385690999 +0000 UTC m=+66.380939289 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qzlcv" (UniqueName: "kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv") pod "network-check-target-zlls4" (UID: "27b91be6-943e-4daf-b4af-aa7947d2993e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 23:10:43.587053 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:43.587015 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:43.587495 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:43.587062 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:43.587495 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:43.587027 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:43.587495 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:43.587151 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4spv8" podUID="84f0d6e1-770c-4014-a4dc-7d27e5a7b1db" Jul 02 23:10:43.587495 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:43.587224 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zz8v" podUID="1352ad10-5c46-4abe-ad7e-9c7d3572be46" Jul 02 23:10:43.587495 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:43.587282 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zlls4" podUID="27b91be6-943e-4daf-b4af-aa7947d2993e" Jul 02 23:10:44.282413 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.282125 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-84.ec2.internal" event="NodeReady" Jul 02 23:10:44.282570 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.282503 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 23:10:44.317691 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.317652 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fcb695cd6-p9pt6"] Jul 02 23:10:44.320059 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.320037 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.322716 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.322698 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 23:10:44.322932 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.322908 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4dq8g\"" Jul 02 23:10:44.323048 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.323021 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 23:10:44.323048 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.323032 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 23:10:44.329397 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.329248 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 23:10:44.333227 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.333177 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qzpwx"] Jul 02 23:10:44.335435 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.335169 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sgl52"] Jul 02 23:10:44.335604 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.335586 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qzpwx" Jul 02 23:10:44.337654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.337631 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fcb695cd6-p9pt6"] Jul 02 23:10:44.337751 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.337716 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.338267 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.338248 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 23:10:44.338361 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.338251 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 23:10:44.338361 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.338284 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 23:10:44.338361 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.338342 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-l7f7m\"" Jul 02 23:10:44.340469 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.340362 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 23:10:44.340469 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.340401 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 23:10:44.340469 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.340406 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bnq4k\"" Jul 02 23:10:44.346025 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.346002 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qzpwx"] Jul 02 23:10:44.354174 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.354150 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sgl52"] Jul 02 23:10:44.502340 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502295 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-bound-sa-token\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.502521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502353 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.502521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502388 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jldkz\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-kube-api-access-jldkz\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.502521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502414 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43611282-80ee-4562-bf56-02fb61dff130-cert\") pod \"ingress-canary-qzpwx\" (UID: \"43611282-80ee-4562-bf56-02fb61dff130\") " pod="openshift-ingress-canary/ingress-canary-qzpwx" Jul 02 23:10:44.502521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502508 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-config-volume\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.502741 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502540 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93c5d449-8ad5-45ef-b99b-afa26ff912d6-ca-trust-extracted\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.502741 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502577 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz7kc\" (UniqueName: \"kubernetes.io/projected/43611282-80ee-4562-bf56-02fb61dff130-kube-api-access-pz7kc\") pod \"ingress-canary-qzpwx\" (UID: \"43611282-80ee-4562-bf56-02fb61dff130\") " pod="openshift-ingress-canary/ingress-canary-qzpwx" Jul 02 23:10:44.502741 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502661 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-image-registry-private-configuration\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.502741 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-trusted-ca\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.502741 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502718 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-installation-pull-secrets\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.502934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502743 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prw85\" (UniqueName: \"kubernetes.io/projected/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-kube-api-access-prw85\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.502934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.502934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-certificates\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.502934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.502890 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-tmp-dir\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.603402 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603366 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-image-registry-private-configuration\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.603402 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603403 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-trusted-ca\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603427 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-installation-pull-secrets\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603451 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prw85\" (UniqueName: \"kubernetes.io/projected/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-kube-api-access-prw85\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603479 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603496 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-certificates\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603524 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-tmp-dir\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603555 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-bound-sa-token\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603574 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603602 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jldkz\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-kube-api-access-jldkz\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43611282-80ee-4562-bf56-02fb61dff130-cert\") pod \"ingress-canary-qzpwx\" (UID: \"43611282-80ee-4562-bf56-02fb61dff130\") " pod="openshift-ingress-canary/ingress-canary-qzpwx" Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:44.603657 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:44.603681 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcb695cd6-p9pt6: secret "image-registry-tls" not found Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:44.603749 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls podName:93c5d449-8ad5-45ef-b99b-afa26ff912d6 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:45.103727869 +0000 UTC m=+37.098976143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls") pod "image-registry-7fcb695cd6-p9pt6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6") : secret "image-registry-tls" not found Jul 02 23:10:44.603927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603880 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-tmp-dir\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.604578 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.603666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-config-volume\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.604578 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.604064 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93c5d449-8ad5-45ef-b99b-afa26ff912d6-ca-trust-extracted\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.604578 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.604100 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz7kc\" (UniqueName: \"kubernetes.io/projected/43611282-80ee-4562-bf56-02fb61dff130-kube-api-access-pz7kc\") pod \"ingress-canary-qzpwx\" (UID: \"43611282-80ee-4562-bf56-02fb61dff130\") " pod="openshift-ingress-canary/ingress-canary-qzpwx" Jul 02 23:10:44.604578 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.604237 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-config-volume\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.604578 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:44.604320 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:10:44.604578 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.604392 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-certificates\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.604578 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:44.604422 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls podName:bc6bbd32-8e79-4ee6-b669-dd27240ddd07 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:45.104403517 +0000 UTC m=+37.099651793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls") pod "dns-default-sgl52" (UID: "bc6bbd32-8e79-4ee6-b669-dd27240ddd07") : secret "dns-default-metrics-tls" not found Jul 02 23:10:44.604886 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.604601 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-trusted-ca\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.604886 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.604689 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93c5d449-8ad5-45ef-b99b-afa26ff912d6-ca-trust-extracted\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.609048 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.609022 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-installation-pull-secrets\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.609048 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.609039 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-image-registry-private-configuration\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.609250 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.609090 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43611282-80ee-4562-bf56-02fb61dff130-cert\") pod \"ingress-canary-qzpwx\" (UID: \"43611282-80ee-4562-bf56-02fb61dff130\") " pod="openshift-ingress-canary/ingress-canary-qzpwx" Jul 02 23:10:44.615626 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.615602 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prw85\" (UniqueName: \"kubernetes.io/projected/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-kube-api-access-prw85\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:44.615781 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.615760 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz7kc\" (UniqueName: \"kubernetes.io/projected/43611282-80ee-4562-bf56-02fb61dff130-kube-api-access-pz7kc\") pod \"ingress-canary-qzpwx\" (UID: \"43611282-80ee-4562-bf56-02fb61dff130\") " pod="openshift-ingress-canary/ingress-canary-qzpwx" Jul 02 23:10:44.616172 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.616125 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-bound-sa-token\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.616322 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.616261 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jldkz\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-kube-api-access-jldkz\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:44.648048 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:44.648014 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qzpwx" Jul 02 23:10:45.107981 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.107945 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:45.108168 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.107990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:45.108168 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:45.108111 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:10:45.108168 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:45.108121 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:10:45.108168 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:45.108128 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcb695cd6-p9pt6: secret "image-registry-tls" not found Jul 02 23:10:45.108363 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:45.108215 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls podName:93c5d449-8ad5-45ef-b99b-afa26ff912d6 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:46.108175346 +0000 UTC m=+38.103423618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls") pod "image-registry-7fcb695cd6-p9pt6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6") : secret "image-registry-tls" not found Jul 02 23:10:45.108363 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:45.108232 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls podName:bc6bbd32-8e79-4ee6-b669-dd27240ddd07 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:46.108225463 +0000 UTC m=+38.103473732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls") pod "dns-default-sgl52" (UID: "bc6bbd32-8e79-4ee6-b669-dd27240ddd07") : secret "dns-default-metrics-tls" not found Jul 02 23:10:45.486927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.486899 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qzpwx"] Jul 02 23:10:45.491135 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:45.491109 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43611282_80ee_4562_bf56_02fb61dff130.slice/crio-b4a82bfa73815490d3ab766bd75f41626d964266d7a0fbf05f9946299bd0de43 WatchSource:0}: Error finding container b4a82bfa73815490d3ab766bd75f41626d964266d7a0fbf05f9946299bd0de43: Status 404 returned error can't find the container with id b4a82bfa73815490d3ab766bd75f41626d964266d7a0fbf05f9946299bd0de43 Jul 02 23:10:45.587270 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.587245 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:10:45.587270 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.587258 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:10:45.587501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.587256 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:10:45.590566 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.590545 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:10:45.590728 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.590705 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:10:45.591845 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.591818 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-flmt2\"" Jul 02 23:10:45.591924 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.591852 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:10:45.591977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.591916 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mp9kj\"" Jul 02 23:10:45.591977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.591956 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:10:45.772398 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.772311 2570 generic.go:356] "Generic (PLEG): container finished" podID="9c2e3cc3-5a74-4577-81ad-4439993bfb17" containerID="1c6c4f6bf6b92bde881a5d6f0b6d1e3653b9fc6045822234ac6e1c2caec39951" exitCode=0 Jul 02 23:10:45.772398 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.772384 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerDied","Data":"1c6c4f6bf6b92bde881a5d6f0b6d1e3653b9fc6045822234ac6e1c2caec39951"} Jul 02 23:10:45.773505 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:45.773485 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qzpwx" event={"ID":"43611282-80ee-4562-bf56-02fb61dff130","Type":"ContainerStarted","Data":"b4a82bfa73815490d3ab766bd75f41626d964266d7a0fbf05f9946299bd0de43"} Jul 02 23:10:46.115836 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:46.115599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:46.116028 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:46.115888 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:46.116028 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:46.115612 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:10:46.116028 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:46.115919 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcb695cd6-p9pt6: secret "image-registry-tls" not found Jul 02 23:10:46.116028 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:46.115986 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls podName:93c5d449-8ad5-45ef-b99b-afa26ff912d6 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:48.115966142 +0000 UTC m=+40.111214431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls") pod "image-registry-7fcb695cd6-p9pt6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6") : secret "image-registry-tls" not found Jul 02 23:10:46.116028 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:46.116017 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:10:46.116320 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:46.116055 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls podName:bc6bbd32-8e79-4ee6-b669-dd27240ddd07 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:48.11604208 +0000 UTC m=+40.111290348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls") pod "dns-default-sgl52" (UID: "bc6bbd32-8e79-4ee6-b669-dd27240ddd07") : secret "dns-default-metrics-tls" not found Jul 02 23:10:46.779961 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:46.779924 2570 generic.go:356] "Generic (PLEG): container finished" podID="9c2e3cc3-5a74-4577-81ad-4439993bfb17" containerID="f106b3e7456b8fd3e54d7b74b854b24206ba083b81d6da72fce0a34bb35b54c5" exitCode=0 Jul 02 23:10:46.780411 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:46.779992 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerDied","Data":"f106b3e7456b8fd3e54d7b74b854b24206ba083b81d6da72fce0a34bb35b54c5"} Jul 02 23:10:47.786802 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:47.786762 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" event={"ID":"9c2e3cc3-5a74-4577-81ad-4439993bfb17","Type":"ContainerStarted","Data":"b2b3054a1f6a020408b0c159eaf6aaf9afc98e05baa75b5d58beab49d7781a42"} Jul 02 23:10:47.825727 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:47.825629 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6qnb6" podStartSLOduration=7.785183967 podStartE2EDuration="38.825525816s" totalImagesPullingTime="31.040341849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:47.824836001 +0000 UTC m=+39.820084343" watchObservedRunningTime="2026-07-02 23:10:47.825525816 +0000 UTC m=+39.820774109" Jul 02 23:10:48.131885 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.131839 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:48.131885 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.131885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:48.132162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:48.132000 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:10:48.132162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:48.132020 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcb695cd6-p9pt6: secret "image-registry-tls" not found Jul 02 23:10:48.132162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:48.132047 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:10:48.132162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:48.132076 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls podName:93c5d449-8ad5-45ef-b99b-afa26ff912d6 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:52.132061113 +0000 UTC m=+44.127309381 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls") pod "image-registry-7fcb695cd6-p9pt6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6") : secret "image-registry-tls" not found Jul 02 23:10:48.132162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:48.132153 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls podName:bc6bbd32-8e79-4ee6-b669-dd27240ddd07 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:52.132135254 +0000 UTC m=+44.127383526 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls") pod "dns-default-sgl52" (UID: "bc6bbd32-8e79-4ee6-b669-dd27240ddd07") : secret "dns-default-metrics-tls" not found Jul 02 23:10:48.736806 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.736762 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6"] Jul 02 23:10:48.755080 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.755047 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h"] Jul 02 23:10:48.755259 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.755173 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:48.762536 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.762497 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 02 23:10:48.762690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.762563 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:10:48.762762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.762731 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 02 23:10:48.763904 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.763885 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 02 23:10:48.763968 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.763885 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9n4k5\"" Jul 02 23:10:48.772784 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.772767 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf"] Jul 02 23:10:48.772888 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.772874 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:48.776381 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.776352 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 02 23:10:48.776472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.776408 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:10:48.776472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.776410 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 02 23:10:48.776472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.776356 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kjqm7\"" Jul 02 23:10:48.788665 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.788596 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6"] Jul 02 23:10:48.788995 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.788710 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf" Jul 02 23:10:48.789254 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.789232 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 02 23:10:48.791100 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.791071 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qzpwx" event={"ID":"43611282-80ee-4562-bf56-02fb61dff130","Type":"ContainerStarted","Data":"a5ace2dffc01887c5a39d4582425c39ad48aa43a9201521e60b9515e76ca29f8"} Jul 02 23:10:48.795472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.795268 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h"] Jul 02 23:10:48.795733 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.795711 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:10:48.795885 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.795866 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zh25p\"" Jul 02 23:10:48.815648 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.815617 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 02 23:10:48.816443 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.816420 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf"] Jul 02 23:10:48.840508 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.840481 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6d4cf774dc-pdxfl"] Jul 02 23:10:48.864727 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.864702 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d4cf774dc-pdxfl"] Jul 02 23:10:48.864854 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.864790 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:48.870020 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.869996 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7jvgc\"" Jul 02 23:10:48.870614 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.870595 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 02 23:10:48.871072 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.871022 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 23:10:48.871072 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.871029 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 23:10:48.871072 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.871058 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 02 23:10:48.871335 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.871314 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 02 23:10:48.872263 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.872244 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 02 23:10:48.917196 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.917137 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qzpwx" podStartSLOduration=1.879226229 podStartE2EDuration="4.917122043s" totalImagesPullingTime="3.037895814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:48.915361497 +0000 UTC m=+40.910609788" watchObservedRunningTime="2026-07-02 23:10:48.917122043 +0000 UTC m=+40.912370334" Jul 02 23:10:48.921862 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.921740 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp"] Jul 02 23:10:48.939729 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.939697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2384cb4b-da67-4522-8ca7-d721551a31bb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:48.939931 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.939904 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7ljk\" (UniqueName: \"kubernetes.io/projected/2384cb4b-da67-4522-8ca7-d721551a31bb-kube-api-access-v7ljk\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:48.940018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.939951 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ce22038-a78d-48ec-824c-1e1ff97400d2-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:48.940018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.939976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pvqw\" (UniqueName: \"kubernetes.io/projected/2ce22038-a78d-48ec-824c-1e1ff97400d2-kube-api-access-4pvqw\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:48.940113 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.940091 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f85v8\" (UniqueName: \"kubernetes.io/projected/5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8-kube-api-access-f85v8\") pod \"volume-data-source-validator-5d456fdbcd-jbpdf\" (UID: \"5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf" Jul 02 23:10:48.940336 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.940262 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2384cb4b-da67-4522-8ca7-d721551a31bb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:48.940434 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.940391 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ce22038-a78d-48ec-824c-1e1ff97400d2-config\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:48.941018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.940994 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp"] Jul 02 23:10:48.941122 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.941096 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:48.944892 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.944870 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 02 23:10:48.946925 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.946576 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 23:10:48.946925 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.946708 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 23:10:48.946925 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.946583 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 02 23:10:48.946925 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:48.946766 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fjnzv\"" Jul 02 23:10:49.041861 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.041775 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-default-certificate\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.041861 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.041836 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrnxg\" (UniqueName: \"kubernetes.io/projected/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-kube-api-access-rrnxg\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.041861 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.041860 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.042078 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.041890 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ce22038-a78d-48ec-824c-1e1ff97400d2-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:49.042078 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.041913 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.042078 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.041929 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.042078 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.041976 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2384cb4b-da67-4522-8ca7-d721551a31bb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:49.042256 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042119 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f85v8\" (UniqueName: \"kubernetes.io/projected/5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8-kube-api-access-f85v8\") pod \"volume-data-source-validator-5d456fdbcd-jbpdf\" (UID: \"5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf" Jul 02 23:10:49.042256 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042149 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9s5d\" (UniqueName: \"kubernetes.io/projected/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-kube-api-access-x9s5d\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.042256 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042171 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pvqw\" (UniqueName: \"kubernetes.io/projected/2ce22038-a78d-48ec-824c-1e1ff97400d2-kube-api-access-4pvqw\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:49.042256 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042208 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-stats-auth\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.042417 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042340 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.042417 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042365 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2384cb4b-da67-4522-8ca7-d721551a31bb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:49.042491 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042425 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ce22038-a78d-48ec-824c-1e1ff97400d2-config\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:49.042491 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042478 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7ljk\" (UniqueName: \"kubernetes.io/projected/2384cb4b-da67-4522-8ca7-d721551a31bb-kube-api-access-v7ljk\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:49.042923 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.042896 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ce22038-a78d-48ec-824c-1e1ff97400d2-config\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:49.045168 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.045140 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2384cb4b-da67-4522-8ca7-d721551a31bb-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:49.045299 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.045283 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ce22038-a78d-48ec-824c-1e1ff97400d2-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:49.053783 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.053762 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f85v8\" (UniqueName: \"kubernetes.io/projected/5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8-kube-api-access-f85v8\") pod \"volume-data-source-validator-5d456fdbcd-jbpdf\" (UID: \"5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf" Jul 02 23:10:49.053848 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.053788 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pvqw\" (UniqueName: \"kubernetes.io/projected/2ce22038-a78d-48ec-824c-1e1ff97400d2-kube-api-access-4pvqw\") pod \"service-ca-operator-5d46d9ffd4-t849h\" (UID: \"2ce22038-a78d-48ec-824c-1e1ff97400d2\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:49.053904 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.053882 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7ljk\" (UniqueName: \"kubernetes.io/projected/2384cb4b-da67-4522-8ca7-d721551a31bb-kube-api-access-v7ljk\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:49.056496 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.056475 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2384cb4b-da67-4522-8ca7-d721551a31bb-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-kb5t6\" (UID: \"2384cb4b-da67-4522-8ca7-d721551a31bb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:49.069705 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.069667 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" Jul 02 23:10:49.082837 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.082814 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" Jul 02 23:10:49.098782 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.098767 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf" Jul 02 23:10:49.144143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.143675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-default-certificate\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.144143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.143739 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrnxg\" (UniqueName: \"kubernetes.io/projected/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-kube-api-access-rrnxg\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.144143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.143785 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.144143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.143830 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.144143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.143856 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.144143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.143891 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9s5d\" (UniqueName: \"kubernetes.io/projected/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-kube-api-access-x9s5d\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.144143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.143916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-stats-auth\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.144143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.143948 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.144795 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.144764 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:10:49.144869 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.144858 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:49.644835516 +0000 UTC m=+41.640083810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : secret "router-metrics-certs-default" not found Jul 02 23:10:49.144928 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.144880 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.145504 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.145274 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:49.645257757 +0000 UTC m=+41.640506047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : configmap references non-existent config key: service-ca.crt Jul 02 23:10:49.145610 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.145511 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:49.145610 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.145565 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls podName:b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:49.645549487 +0000 UTC m=+41.640797761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xq5vp" (UID: "b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:49.147800 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.147775 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-default-certificate\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.148587 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.148568 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-stats-auth\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.159039 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.158319 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrnxg\" (UniqueName: \"kubernetes.io/projected/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-kube-api-access-rrnxg\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.159980 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.159926 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9s5d\" (UniqueName: \"kubernetes.io/projected/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-kube-api-access-x9s5d\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.248654 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.248582 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6"] Jul 02 23:10:49.254216 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:49.253841 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2384cb4b_da67_4522_8ca7_d721551a31bb.slice/crio-4aac9a4d4bd998fc16f0d2a6e7e128e1cd9fdf79382cb9d0d633ab013cf80753 WatchSource:0}: Error finding container 4aac9a4d4bd998fc16f0d2a6e7e128e1cd9fdf79382cb9d0d633ab013cf80753: Status 404 returned error can't find the container with id 4aac9a4d4bd998fc16f0d2a6e7e128e1cd9fdf79382cb9d0d633ab013cf80753 Jul 02 23:10:49.281699 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.281585 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h"] Jul 02 23:10:49.284016 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:49.283991 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce22038_a78d_48ec_824c_1e1ff97400d2.slice/crio-7a0f7e47562b1470926aa54819065055c329ef6c4947b7becac983712c56cece WatchSource:0}: Error finding container 7a0f7e47562b1470926aa54819065055c329ef6c4947b7becac983712c56cece: Status 404 returned error can't find the container with id 7a0f7e47562b1470926aa54819065055c329ef6c4947b7becac983712c56cece Jul 02 23:10:49.296956 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.296903 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf"] Jul 02 23:10:49.299859 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:10:49.299837 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aad2a96_277e_4f9c_9f91_dd4eafa2a0b8.slice/crio-c5e2567d41188bb474df222fa32e9a8f45d3c493db8cd13e10b03e8d133dadf8 WatchSource:0}: Error finding container c5e2567d41188bb474df222fa32e9a8f45d3c493db8cd13e10b03e8d133dadf8: Status 404 returned error can't find the container with id c5e2567d41188bb474df222fa32e9a8f45d3c493db8cd13e10b03e8d133dadf8 Jul 02 23:10:49.648415 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.648372 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.648644 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.648423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:49.648644 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.648524 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:49.648644 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.648526 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:10:49.648644 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.648585 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:49.648644 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.648591 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:50.648574068 +0000 UTC m=+42.643822337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : secret "router-metrics-certs-default" not found Jul 02 23:10:49.648833 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.648663 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:50.648648882 +0000 UTC m=+42.643897154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : configmap references non-existent config key: service-ca.crt Jul 02 23:10:49.648833 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:49.648684 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls podName:b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:50.648674329 +0000 UTC m=+42.643922601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xq5vp" (UID: "b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:49.795353 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.795313 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" event={"ID":"2ce22038-a78d-48ec-824c-1e1ff97400d2","Type":"ContainerStarted","Data":"7a0f7e47562b1470926aa54819065055c329ef6c4947b7becac983712c56cece"} Jul 02 23:10:49.796740 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.796709 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" event={"ID":"2384cb4b-da67-4522-8ca7-d721551a31bb","Type":"ContainerStarted","Data":"4aac9a4d4bd998fc16f0d2a6e7e128e1cd9fdf79382cb9d0d633ab013cf80753"} Jul 02 23:10:49.797937 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:49.797915 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf" event={"ID":"5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8","Type":"ContainerStarted","Data":"c5e2567d41188bb474df222fa32e9a8f45d3c493db8cd13e10b03e8d133dadf8"} Jul 02 23:10:50.656721 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:50.656681 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:50.656920 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:50.656751 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:50.656920 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:50.656771 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:50.656920 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:50.656853 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:52.656830534 +0000 UTC m=+44.652078817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : configmap references non-existent config key: service-ca.crt Jul 02 23:10:50.656920 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:50.656859 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:10:50.656920 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:50.656912 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:50.657175 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:50.656958 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:52.656936476 +0000 UTC m=+44.652184769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : secret "router-metrics-certs-default" not found Jul 02 23:10:50.657175 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:50.656979 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls podName:b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:52.656969212 +0000 UTC m=+44.652217481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xq5vp" (UID: "b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:52.168658 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.168621 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:10:52.168658 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.168663 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:10:52.169179 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.168770 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:10:52.169179 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.168831 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls podName:bc6bbd32-8e79-4ee6-b669-dd27240ddd07 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.168816785 +0000 UTC m=+52.164065054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls") pod "dns-default-sgl52" (UID: "bc6bbd32-8e79-4ee6-b669-dd27240ddd07") : secret "dns-default-metrics-tls" not found Jul 02 23:10:52.169179 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.168775 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:10:52.169179 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.168865 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcb695cd6-p9pt6: secret "image-registry-tls" not found Jul 02 23:10:52.169179 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.168913 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls podName:93c5d449-8ad5-45ef-b99b-afa26ff912d6 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.168898835 +0000 UTC m=+52.164147107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls") pod "image-registry-7fcb695cd6-p9pt6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6") : secret "image-registry-tls" not found Jul 02 23:10:52.672801 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.672764 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:52.672801 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.672814 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:52.673214 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.672842 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:52.673214 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.672947 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:56.672926766 +0000 UTC m=+48.668175056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : configmap references non-existent config key: service-ca.crt Jul 02 23:10:52.673214 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.672947 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:52.673214 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.672953 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:10:52.673214 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.673037 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls podName:b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:56.673015302 +0000 UTC m=+48.668263585 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xq5vp" (UID: "b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:52.673214 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:52.673058 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:56.673049598 +0000 UTC m=+48.668297870 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : secret "router-metrics-certs-default" not found Jul 02 23:10:52.808670 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.808583 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" event={"ID":"2384cb4b-da67-4522-8ca7-d721551a31bb","Type":"ContainerStarted","Data":"e511137be9bf3d4a47e311b198d11b5524857a6fa2a142532e7aaa0694ee1460"} Jul 02 23:10:52.810167 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.810137 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf" event={"ID":"5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8","Type":"ContainerStarted","Data":"71621a5f1e18d6ed960ed631175fd76a57962ba1e01fa5c462fd59b1ad02dd05"} Jul 02 23:10:52.811606 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.811586 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" event={"ID":"2ce22038-a78d-48ec-824c-1e1ff97400d2","Type":"ContainerStarted","Data":"ad46eea139bf108f49aeefab5af7809bc7fced8b1f9c93abbab7c42c32fcf69e"} Jul 02 23:10:52.826823 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.826743 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" podStartSLOduration=1.6948986019999999 podStartE2EDuration="4.826727933s" totalImagesPullingTime="3.131829331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:52.8262035 +0000 UTC m=+44.821451793" watchObservedRunningTime="2026-07-02 23:10:52.826727933 +0000 UTC m=+44.821976226" Jul 02 23:10:52.846259 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.846205 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" podStartSLOduration=1.7377683689999999 podStartE2EDuration="4.846170418s" totalImagesPullingTime="3.108402049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:52.845027029 +0000 UTC m=+44.840275346" watchObservedRunningTime="2026-07-02 23:10:52.846170418 +0000 UTC m=+44.841418708" Jul 02 23:10:52.869901 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:52.869853 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-jbpdf" podStartSLOduration=1.8077078850000001 podStartE2EDuration="4.869834689s" totalImagesPullingTime="3.062126804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:10:52.869597849 +0000 UTC m=+44.864846142" watchObservedRunningTime="2026-07-02 23:10:52.869834689 +0000 UTC m=+44.865082979" Jul 02 23:10:56.305122 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.305082 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bzgbm"] Jul 02 23:10:56.328293 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.328256 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bzgbm"] Jul 02 23:10:56.328438 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.328398 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.333244 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.333223 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xgtgg\"" Jul 02 23:10:56.333682 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.333663 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 23:10:56.334703 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.334508 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 23:10:56.334703 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.334549 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 23:10:56.334862 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.334847 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 23:10:56.402589 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.402560 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpmhm\" (UniqueName: \"kubernetes.io/projected/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-api-access-gpmhm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.402754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.402631 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.402754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.402654 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.402754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.402683 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acdd4d3d-68f6-427a-9c96-40384b99b3b8-crio-socket\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.402862 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.402795 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acdd4d3d-68f6-427a-9c96-40384b99b3b8-data-volume\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.504126 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.504088 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acdd4d3d-68f6-427a-9c96-40384b99b3b8-data-volume\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.504126 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.504132 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpmhm\" (UniqueName: \"kubernetes.io/projected/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-api-access-gpmhm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.504407 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.504209 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.504407 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.504228 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.504407 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.504256 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acdd4d3d-68f6-427a-9c96-40384b99b3b8-crio-socket\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.504407 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.504312 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:10:56.504407 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.504352 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:10:56.504407 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.504394 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:57.004371953 +0000 UTC m=+48.999620239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : configmap "kube-rbac-proxy" not found Jul 02 23:10:56.504714 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.504414 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:57.004404846 +0000 UTC m=+48.999653114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : secret "insights-runtime-extractor-tls" not found Jul 02 23:10:56.504714 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.504476 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/acdd4d3d-68f6-427a-9c96-40384b99b3b8-crio-socket\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.504714 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.504483 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/acdd4d3d-68f6-427a-9c96-40384b99b3b8-data-volume\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.513449 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.513428 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpmhm\" (UniqueName: \"kubernetes.io/projected/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-api-access-gpmhm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:56.705999 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.705962 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:56.705999 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.706005 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:10:56.706298 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:56.706024 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:10:56.706298 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.706125 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:56.706298 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.706126 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:10:56.706298 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.706175 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.706153936 +0000 UTC m=+56.701402222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : configmap references non-existent config key: service-ca.crt Jul 02 23:10:56.706298 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.706229 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls podName:b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.706219297 +0000 UTC m=+56.701467566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xq5vp" (UID: "b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:10:56.706298 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:56.706247 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.706237183 +0000 UTC m=+56.701485455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : secret "router-metrics-certs-default" not found Jul 02 23:10:57.008035 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:57.007945 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:57.008035 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:57.007992 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:57.008277 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:57.008073 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:10:57.008277 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:57.008110 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:10:57.008277 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:57.008140 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:58.008121179 +0000 UTC m=+50.003369469 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : configmap "kube-rbac-proxy" not found Jul 02 23:10:57.008277 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:57.008157 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:58.008148168 +0000 UTC m=+50.003396439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : secret "insights-runtime-extractor-tls" not found Jul 02 23:10:58.017508 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:58.017472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:58.017508 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:58.017506 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:10:58.017913 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:58.017593 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:10:58.017913 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:58.017609 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:10:58.017913 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:58.017659 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.017646184 +0000 UTC m=+52.012894454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : secret "insights-runtime-extractor-tls" not found Jul 02 23:10:58.017913 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:10:58.017673 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.017666987 +0000 UTC m=+52.012915256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : configmap "kube-rbac-proxy" not found Jul 02 23:10:59.086320 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:10:59.086288 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mdw9r_5ef83259-01b2-4ae4-bdbc-af8081c9fa8a/dns-node-resolver/0.log" Jul 02 23:11:00.036651 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:00.036614 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:00.036651 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:00.036649 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:00.036878 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.036738 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:11:00.036878 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.036767 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:11:00.036878 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.036796 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.036780699 +0000 UTC m=+56.032028968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : configmap "kube-rbac-proxy" not found Jul 02 23:11:00.036993 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.036889 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.036873246 +0000 UTC m=+56.032121515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : secret "insights-runtime-extractor-tls" not found Jul 02 23:11:00.086526 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:00.086489 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fv8lb_55620e5e-72b6-4a11-afd9-f85e0171ba51/node-ca/0.log" Jul 02 23:11:00.238922 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:00.238885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:11:00.239059 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:00.238936 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:11:00.239059 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.239031 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 23:11:00.239162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.239094 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls podName:bc6bbd32-8e79-4ee6-b669-dd27240ddd07 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:16.239077918 +0000 UTC m=+68.234326186 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls") pod "dns-default-sgl52" (UID: "bc6bbd32-8e79-4ee6-b669-dd27240ddd07") : secret "dns-default-metrics-tls" not found Jul 02 23:11:00.239162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.239037 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 23:11:00.239162 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.239134 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fcb695cd6-p9pt6: secret "image-registry-tls" not found Jul 02 23:11:00.239286 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:00.239197 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls podName:93c5d449-8ad5-45ef-b99b-afa26ff912d6 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:16.23916567 +0000 UTC m=+68.234413947 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls") pod "image-registry-7fcb695cd6-p9pt6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6") : secret "image-registry-tls" not found Jul 02 23:11:00.903952 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:00.903926 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qzpwx_43611282-80ee-4562-bf56-02fb61dff130/serve-healthcheck-canary/0.log" Jul 02 23:11:01.687706 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:01.687681 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-kb5t6_2384cb4b-da67-4522-8ca7-d721551a31bb/kube-storage-version-migrator-operator/0.log" Jul 02 23:11:04.072256 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:04.072215 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:04.072256 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:04.072257 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:04.072808 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.072339 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 23:11:04.072808 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.072407 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:12.072393526 +0000 UTC m=+64.067641798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : configmap "kube-rbac-proxy" not found Jul 02 23:11:04.072808 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.072448 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 23:11:04.072808 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.072509 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls podName:acdd4d3d-68f6-427a-9c96-40384b99b3b8 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:12.072495827 +0000 UTC m=+64.067744096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bzgbm" (UID: "acdd4d3d-68f6-427a-9c96-40384b99b3b8") : secret "insights-runtime-extractor-tls" not found Jul 02 23:11:04.778233 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:04.778173 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:04.778233 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:04.778242 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:04.778594 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:04.778259 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:11:04.778594 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.778353 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:04.778594 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.778357 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:20.778338398 +0000 UTC m=+72.773586672 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : configmap references non-existent config key: service-ca.crt Jul 02 23:11:04.778594 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.778401 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls podName:b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:20.778391334 +0000 UTC m=+72.773639603 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xq5vp" (UID: "b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9") : secret "cluster-monitoring-operator-tls" not found Jul 02 23:11:04.778594 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.778430 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 23:11:04.778594 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:04.778511 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs podName:ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:20.77849448 +0000 UTC m=+72.773742750 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs") pod "router-default-6d4cf774dc-pdxfl" (UID: "ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1") : secret "router-metrics-certs-default" not found Jul 02 23:11:08.759534 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:08.759507 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g6gs8" Jul 02 23:11:12.141341 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:12.141306 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:12.141341 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:12.141344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:12.141851 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:12.141829 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/acdd4d3d-68f6-427a-9c96-40384b99b3b8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:12.143598 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:12.143581 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/acdd4d3d-68f6-427a-9c96-40384b99b3b8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bzgbm\" (UID: \"acdd4d3d-68f6-427a-9c96-40384b99b3b8\") " pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:12.239934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:12.239902 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xgtgg\"" Jul 02 23:11:12.247225 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:12.247179 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bzgbm" Jul 02 23:11:12.367871 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:12.367836 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bzgbm"] Jul 02 23:11:12.370876 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:12.370851 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacdd4d3d_68f6_427a_9c96_40384b99b3b8.slice/crio-7f1455f5e1f54cb49409ce1f5a0c01b5802d89c73f7d2ebd57533a97a8f26ff9 WatchSource:0}: Error finding container 7f1455f5e1f54cb49409ce1f5a0c01b5802d89c73f7d2ebd57533a97a8f26ff9: Status 404 returned error can't find the container with id 7f1455f5e1f54cb49409ce1f5a0c01b5802d89c73f7d2ebd57533a97a8f26ff9 Jul 02 23:11:12.865839 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:12.865796 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bzgbm" event={"ID":"acdd4d3d-68f6-427a-9c96-40384b99b3b8","Type":"ContainerStarted","Data":"7f1455f5e1f54cb49409ce1f5a0c01b5802d89c73f7d2ebd57533a97a8f26ff9"} Jul 02 23:11:14.260627 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.260587 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:11:14.263955 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.263924 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 23:11:14.273996 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.273968 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1352ad10-5c46-4abe-ad7e-9c7d3572be46-original-pull-secret\") pod \"global-pull-secret-syncer-6zz8v\" (UID: \"1352ad10-5c46-4abe-ad7e-9c7d3572be46\") " pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:11:14.361811 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.361771 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:11:14.365304 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.365277 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 23:11:14.375110 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.375078 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84f0d6e1-770c-4014-a4dc-7d27e5a7b1db-metrics-certs\") pod \"network-metrics-daemon-4spv8\" (UID: \"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db\") " pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:11:14.427961 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.427931 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-flmt2\"" Jul 02 23:11:14.429970 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.429948 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zz8v" Jul 02 23:11:14.435581 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.435558 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4spv8" Jul 02 23:11:14.462762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.462723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:11:14.466196 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.466156 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 23:11:14.477378 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.477349 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 23:11:14.486682 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.486659 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzlcv\" (UniqueName: \"kubernetes.io/projected/27b91be6-943e-4daf-b4af-aa7947d2993e-kube-api-access-qzlcv\") pod \"network-check-target-zlls4\" (UID: \"27b91be6-943e-4daf-b4af-aa7947d2993e\") " pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:11:14.723004 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.722975 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mp9kj\"" Jul 02 23:11:14.729926 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:14.729898 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:11:15.698227 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.698177 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-4z9jb"] Jul 02 23:11:15.700519 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.700496 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-4z9jb" Jul 02 23:11:15.705353 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.705325 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 23:11:15.705728 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.705688 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 23:11:15.706514 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.706496 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9w77x\"" Jul 02 23:11:15.721598 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.721527 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-4z9jb"] Jul 02 23:11:15.776434 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.776406 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntv9t\" (UniqueName: \"kubernetes.io/projected/5da6a02f-66c5-4af3-b26f-16fb14e7ff2a-kube-api-access-ntv9t\") pod \"downloads-685bfb5f64-4z9jb\" (UID: \"5da6a02f-66c5-4af3-b26f-16fb14e7ff2a\") " pod="openshift-console/downloads-685bfb5f64-4z9jb" Jul 02 23:11:15.877140 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.877075 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntv9t\" (UniqueName: \"kubernetes.io/projected/5da6a02f-66c5-4af3-b26f-16fb14e7ff2a-kube-api-access-ntv9t\") pod \"downloads-685bfb5f64-4z9jb\" (UID: \"5da6a02f-66c5-4af3-b26f-16fb14e7ff2a\") " pod="openshift-console/downloads-685bfb5f64-4z9jb" Jul 02 23:11:15.887305 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:15.887275 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntv9t\" (UniqueName: \"kubernetes.io/projected/5da6a02f-66c5-4af3-b26f-16fb14e7ff2a-kube-api-access-ntv9t\") pod \"downloads-685bfb5f64-4z9jb\" (UID: \"5da6a02f-66c5-4af3-b26f-16fb14e7ff2a\") " pod="openshift-console/downloads-685bfb5f64-4z9jb" Jul 02 23:11:16.011573 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.011381 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-4z9jb" Jul 02 23:11:16.023670 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.023591 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6zz8v"] Jul 02 23:11:16.040076 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.040046 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4spv8"] Jul 02 23:11:16.044161 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:16.044129 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f0d6e1_770c_4014_a4dc_7d27e5a7b1db.slice/crio-45be131db03386a4f2453ebcf23769aa40e046afc36b3c7e4238070df2e00a15 WatchSource:0}: Error finding container 45be131db03386a4f2453ebcf23769aa40e046afc36b3c7e4238070df2e00a15: Status 404 returned error can't find the container with id 45be131db03386a4f2453ebcf23769aa40e046afc36b3c7e4238070df2e00a15 Jul 02 23:11:16.063269 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.063230 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zlls4"] Jul 02 23:11:16.066312 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:16.066273 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b91be6_943e_4daf_b4af_aa7947d2993e.slice/crio-9a472dac057fc461eb2d6bd7474b839a5e7c8e929fca80fccac7ff796a9cd9c2 WatchSource:0}: Error finding container 9a472dac057fc461eb2d6bd7474b839a5e7c8e929fca80fccac7ff796a9cd9c2: Status 404 returned error can't find the container with id 9a472dac057fc461eb2d6bd7474b839a5e7c8e929fca80fccac7ff796a9cd9c2 Jul 02 23:11:16.159396 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.159365 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-4z9jb"] Jul 02 23:11:16.165807 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:16.165780 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da6a02f_66c5_4af3_b26f_16fb14e7ff2a.slice/crio-fbc18ce6b189549e3ff6febea2b6958dc64d402b9ec8eb3c9b22ffe425db2257 WatchSource:0}: Error finding container fbc18ce6b189549e3ff6febea2b6958dc64d402b9ec8eb3c9b22ffe425db2257: Status 404 returned error can't find the container with id fbc18ce6b189549e3ff6febea2b6958dc64d402b9ec8eb3c9b22ffe425db2257 Jul 02 23:11:16.281866 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.281781 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:11:16.281866 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.281827 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:11:16.284201 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.284164 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6bbd32-8e79-4ee6-b669-dd27240ddd07-metrics-tls\") pod \"dns-default-sgl52\" (UID: \"bc6bbd32-8e79-4ee6-b669-dd27240ddd07\") " pod="openshift-dns/dns-default-sgl52" Jul 02 23:11:16.284320 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.284209 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"image-registry-7fcb695cd6-p9pt6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:11:16.434933 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.434902 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4dq8g\"" Jul 02 23:11:16.443273 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.443247 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:11:16.458789 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.458764 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bnq4k\"" Jul 02 23:11:16.467149 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.467125 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sgl52" Jul 02 23:11:16.663847 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.661556 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fcb695cd6-p9pt6"] Jul 02 23:11:16.666960 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:16.666932 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c5d449_8ad5_45ef_b99b_afa26ff912d6.slice/crio-9117a7c737d25cf21767fab9c6bdff7a225cabc3cfece268f7e1899b79b5054d WatchSource:0}: Error finding container 9117a7c737d25cf21767fab9c6bdff7a225cabc3cfece268f7e1899b79b5054d: Status 404 returned error can't find the container with id 9117a7c737d25cf21767fab9c6bdff7a225cabc3cfece268f7e1899b79b5054d Jul 02 23:11:16.675587 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.675505 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sgl52"] Jul 02 23:11:16.679474 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:16.679452 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6bbd32_8e79_4ee6_b669_dd27240ddd07.slice/crio-4b83cc7aaa09c08a0493e12c986db253b20c0ef262908158ee217ebc324957e9 WatchSource:0}: Error finding container 4b83cc7aaa09c08a0493e12c986db253b20c0ef262908158ee217ebc324957e9: Status 404 returned error can't find the container with id 4b83cc7aaa09c08a0493e12c986db253b20c0ef262908158ee217ebc324957e9 Jul 02 23:11:16.882005 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.881911 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4spv8" event={"ID":"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db","Type":"ContainerStarted","Data":"45be131db03386a4f2453ebcf23769aa40e046afc36b3c7e4238070df2e00a15"} Jul 02 23:11:16.885156 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.884950 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bzgbm" event={"ID":"acdd4d3d-68f6-427a-9c96-40384b99b3b8","Type":"ContainerStarted","Data":"822cec50a26607a569420bd1acf2bab0b0de5c91ac76b639c80767365bf796d7"} Jul 02 23:11:16.885156 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.885011 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bzgbm" event={"ID":"acdd4d3d-68f6-427a-9c96-40384b99b3b8","Type":"ContainerStarted","Data":"a53fc6fee30ff96423ac0dc5819e7f23a5926c8c7a873491d910f528a62830cd"} Jul 02 23:11:16.889126 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.889057 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sgl52" event={"ID":"bc6bbd32-8e79-4ee6-b669-dd27240ddd07","Type":"ContainerStarted","Data":"4b83cc7aaa09c08a0493e12c986db253b20c0ef262908158ee217ebc324957e9"} Jul 02 23:11:16.891677 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.890819 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" event={"ID":"93c5d449-8ad5-45ef-b99b-afa26ff912d6","Type":"ContainerStarted","Data":"c0bed6161870776de78e61ed8549a5fa7f1e325f6808b925d917069c1f8bd9ff"} Jul 02 23:11:16.891677 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.890862 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" event={"ID":"93c5d449-8ad5-45ef-b99b-afa26ff912d6","Type":"ContainerStarted","Data":"9117a7c737d25cf21767fab9c6bdff7a225cabc3cfece268f7e1899b79b5054d"} Jul 02 23:11:16.891677 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.891636 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:11:16.895348 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.895285 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-4z9jb" event={"ID":"5da6a02f-66c5-4af3-b26f-16fb14e7ff2a","Type":"ContainerStarted","Data":"fbc18ce6b189549e3ff6febea2b6958dc64d402b9ec8eb3c9b22ffe425db2257"} Jul 02 23:11:16.899577 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.899513 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6zz8v" event={"ID":"1352ad10-5c46-4abe-ad7e-9c7d3572be46","Type":"ContainerStarted","Data":"b8cced8b1300f27642e7dbd0cb266973320a3d72cfaefaa19e9138594e4d660e"} Jul 02 23:11:16.902609 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.902557 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zlls4" event={"ID":"27b91be6-943e-4daf-b4af-aa7947d2993e","Type":"ContainerStarted","Data":"9a472dac057fc461eb2d6bd7474b839a5e7c8e929fca80fccac7ff796a9cd9c2"} Jul 02 23:11:16.924523 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:16.924126 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" podStartSLOduration=67.924104403 podStartE2EDuration="1m7.924104403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:16.922902459 +0000 UTC m=+68.918150761" watchObservedRunningTime="2026-07-02 23:11:16.924104403 +0000 UTC m=+68.919352694" Jul 02 23:11:18.919780 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:18.919638 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4spv8" event={"ID":"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db","Type":"ContainerStarted","Data":"3de8a086c2e7952af15359ef7d60f2564c8b768232c3419112d1ef4139617a07"} Jul 02 23:11:18.919780 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:18.919730 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4spv8" event={"ID":"84f0d6e1-770c-4014-a4dc-7d27e5a7b1db","Type":"ContainerStarted","Data":"4f57af17e28276f9d352b36231317929ebf9c252c14b1afbcf94de2efae5e21e"} Jul 02 23:11:18.939087 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:18.939020 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4spv8" podStartSLOduration=67.79816868 podStartE2EDuration="1m9.938999751s" totalImagesPullingTime="2.140831071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:18.937053501 +0000 UTC m=+70.932301793" watchObservedRunningTime="2026-07-02 23:11:18.938999751 +0000 UTC m=+70.934248046" Jul 02 23:11:20.825526 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:20.825483 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:20.826110 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:20.825545 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:20.826110 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:20.825572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:11:20.826315 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:20.826245 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-service-ca-bundle\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:20.828528 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:20.828502 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1-metrics-certs\") pod \"router-default-6d4cf774dc-pdxfl\" (UID: \"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1\") " pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:20.828690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:20.828646 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xq5vp\" (UID: \"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:11:20.976585 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:20.976550 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7jvgc\"" Jul 02 23:11:20.984970 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:20.984864 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:21.052886 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:21.052849 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fjnzv\"" Jul 02 23:11:21.060406 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:21.060366 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" Jul 02 23:11:23.852610 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.852502 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d4cf774dc-pdxfl"] Jul 02 23:11:23.858166 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:23.858130 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0fad6c_5cdb_4e12_9fa8_305bdd3466e1.slice/crio-2813f041e7823d7c9ccd108163c4ec3aa3ea34e0f10ee4426ecf7b4a99d5e501 WatchSource:0}: Error finding container 2813f041e7823d7c9ccd108163c4ec3aa3ea34e0f10ee4426ecf7b4a99d5e501: Status 404 returned error can't find the container with id 2813f041e7823d7c9ccd108163c4ec3aa3ea34e0f10ee4426ecf7b4a99d5e501 Jul 02 23:11:23.866428 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.866358 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp"] Jul 02 23:11:23.875501 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:23.875469 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb735b6c9_3f1e_4d8c_b5fc_a8b4ca5ddca9.slice/crio-b4bff78ba76383eb5578948c5b7a1acf040f3c585038ac93f81848a982db5ea3 WatchSource:0}: Error finding container b4bff78ba76383eb5578948c5b7a1acf040f3c585038ac93f81848a982db5ea3: Status 404 returned error can't find the container with id b4bff78ba76383eb5578948c5b7a1acf040f3c585038ac93f81848a982db5ea3 Jul 02 23:11:23.945160 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.945008 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zlls4" event={"ID":"27b91be6-943e-4daf-b4af-aa7947d2993e","Type":"ContainerStarted","Data":"e51fc545103665ddbc7ca729a2670d9ed2a0e1a7b13bd5f00bbb2d4a92c0682c"} Jul 02 23:11:23.945899 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.945789 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:11:23.947540 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.947479 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" event={"ID":"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9","Type":"ContainerStarted","Data":"b4bff78ba76383eb5578948c5b7a1acf040f3c585038ac93f81848a982db5ea3"} Jul 02 23:11:23.950218 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.950155 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" event={"ID":"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1","Type":"ContainerStarted","Data":"3a0ec6ca55e5eea0a1df3143072d3fb3edc8453feaa8d0d3f30fc266e81317d1"} Jul 02 23:11:23.950323 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.950219 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" event={"ID":"ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1","Type":"ContainerStarted","Data":"2813f041e7823d7c9ccd108163c4ec3aa3ea34e0f10ee4426ecf7b4a99d5e501"} Jul 02 23:11:23.953222 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.953149 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bzgbm" event={"ID":"acdd4d3d-68f6-427a-9c96-40384b99b3b8","Type":"ContainerStarted","Data":"7767c15f89e0da6eb4b369c6d360d9605f8cb6427634c641bc3775ecf7ded7f1"} Jul 02 23:11:23.956930 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.956672 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sgl52" event={"ID":"bc6bbd32-8e79-4ee6-b669-dd27240ddd07","Type":"ContainerStarted","Data":"aaa0a3433da0a722a5215a414061c22ac8e3b579ad68fc1bcec3ee9fe42f3a89"} Jul 02 23:11:23.956930 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.956887 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sgl52" Jul 02 23:11:23.959474 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.959067 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6zz8v" event={"ID":"1352ad10-5c46-4abe-ad7e-9c7d3572be46","Type":"ContainerStarted","Data":"b51e6f6766b4a9f1e8bfb93425f6d386132f64c46e2a6315a78a98810a6bd708"} Jul 02 23:11:23.964314 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.964230 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zlls4" podStartSLOduration=67.364447306 podStartE2EDuration="1m14.96421542s" totalImagesPullingTime="7.599768114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:23.962950443 +0000 UTC m=+75.958198767" watchObservedRunningTime="2026-07-02 23:11:23.96421542 +0000 UTC m=+75.959463711" Jul 02 23:11:23.985107 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.985064 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:23.989087 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.988101 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" podStartSLOduration=35.988079659 podStartE2EDuration="35.988079659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:23.987220712 +0000 UTC m=+75.982469026" watchObservedRunningTime="2026-07-02 23:11:23.988079659 +0000 UTC m=+75.983327949" Jul 02 23:11:23.989513 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.989441 2570 patch_prober.go:28] interesting pod/router-default-6d4cf774dc-pdxfl container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 02 23:11:23.989513 ip-10-0-136-84 kubenswrapper[2570]: [-]has-synced failed: reason withheld Jul 02 23:11:23.989513 ip-10-0-136-84 kubenswrapper[2570]: [+]process-running ok Jul 02 23:11:23.989513 ip-10-0-136-84 kubenswrapper[2570]: healthz check failed Jul 02 23:11:23.989513 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:23.989505 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" podUID="ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 02 23:11:24.007159 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:24.007097 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sgl52" podStartSLOduration=33.026358561 podStartE2EDuration="40.007080696s" totalImagesPullingTime="6.980722135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:24.005342231 +0000 UTC m=+76.000590548" watchObservedRunningTime="2026-07-02 23:11:24.007080696 +0000 UTC m=+76.002328988" Jul 02 23:11:24.027164 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:24.027105 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6zz8v" podStartSLOduration=67.378349157 podStartE2EDuration="1m15.027083417s" totalImagesPullingTime="7.64873426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:24.026802887 +0000 UTC m=+76.022051181" watchObservedRunningTime="2026-07-02 23:11:24.027083417 +0000 UTC m=+76.022331711" Jul 02 23:11:24.045824 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:24.045737 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bzgbm" podStartSLOduration=16.920549938 podStartE2EDuration="28.045718467s" totalImagesPullingTime="11.125168529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:24.044437255 +0000 UTC m=+76.039685546" watchObservedRunningTime="2026-07-02 23:11:24.045718467 +0000 UTC m=+76.040966760" Jul 02 23:11:24.965382 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:24.965339 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sgl52" event={"ID":"bc6bbd32-8e79-4ee6-b669-dd27240ddd07","Type":"ContainerStarted","Data":"bd322f638869fb6650426a08199d5b5ec0841be253ccef55103f11a32ddc467d"} Jul 02 23:11:24.987804 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:24.987753 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:25.808776 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.808744 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58d84559d4-rh27t"] Jul 02 23:11:25.823726 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.823698 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d84559d4-rh27t"] Jul 02 23:11:25.823882 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.823800 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:25.827316 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.827288 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 23:11:25.827316 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.827308 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 23:11:25.827544 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.827308 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7npfp\"" Jul 02 23:11:25.827544 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.827288 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 23:11:25.828400 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.828377 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 23:11:25.828400 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.828395 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 23:11:25.966961 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.966925 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5m9j\" (UniqueName: \"kubernetes.io/projected/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-kube-api-access-b5m9j\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:25.966961 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.966968 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-config\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:25.967516 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.966997 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-oauth-config\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:25.967516 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.967054 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-service-ca\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:25.967516 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.967110 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-serving-cert\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:25.967516 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.967147 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-oauth-serving-cert\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:25.969606 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.969579 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:25.970570 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:25.970550 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6d4cf774dc-pdxfl" Jul 02 23:11:26.068209 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.067526 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-service-ca\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.068209 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.067686 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-serving-cert\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.068209 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.067740 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-oauth-serving-cert\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.068209 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.067799 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5m9j\" (UniqueName: \"kubernetes.io/projected/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-kube-api-access-b5m9j\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.068209 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.067828 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-config\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.068209 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.067884 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-oauth-config\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.068728 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.068284 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-service-ca\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.068728 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.068604 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-oauth-serving-cert\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.069674 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.069648 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-config\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.071946 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.071927 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-oauth-config\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.073373 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.073350 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-serving-cert\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.090626 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.090592 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5m9j\" (UniqueName: \"kubernetes.io/projected/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-kube-api-access-b5m9j\") pod \"console-58d84559d4-rh27t\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.134300 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.134267 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:26.347693 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.347645 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58d84559d4-rh27t"] Jul 02 23:11:26.351487 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:26.351457 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7ca2b1_5f0a_4016_8661_40f6c87f54ff.slice/crio-9f2d3c5308f06ce88581d960f9e9b73481e69597a5e3240b74dccc209dc2a459 WatchSource:0}: Error finding container 9f2d3c5308f06ce88581d960f9e9b73481e69597a5e3240b74dccc209dc2a459: Status 404 returned error can't find the container with id 9f2d3c5308f06ce88581d960f9e9b73481e69597a5e3240b74dccc209dc2a459 Jul 02 23:11:26.630268 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.630152 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb"] Jul 02 23:11:26.632724 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.632700 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" Jul 02 23:11:26.635602 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.635427 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-l4jrn\"" Jul 02 23:11:26.635602 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.635472 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 23:11:26.644716 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.644657 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb"] Jul 02 23:11:26.774897 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.774861 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c64aa82-5448-4089-9fd3-9e89dd921a9b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n9rnb\" (UID: \"5c64aa82-5448-4089-9fd3-9e89dd921a9b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" Jul 02 23:11:26.875339 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.875302 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c64aa82-5448-4089-9fd3-9e89dd921a9b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n9rnb\" (UID: \"5c64aa82-5448-4089-9fd3-9e89dd921a9b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" Jul 02 23:11:26.878073 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.878051 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c64aa82-5448-4089-9fd3-9e89dd921a9b-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-n9rnb\" (UID: \"5c64aa82-5448-4089-9fd3-9e89dd921a9b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" Jul 02 23:11:26.943445 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.943362 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" Jul 02 23:11:26.975014 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.974974 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" event={"ID":"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9","Type":"ContainerStarted","Data":"1c03411cc20f23e9fe225e6bf34cba1e632c218badcbadd5e3ff3d34ec890fb6"} Jul 02 23:11:26.976503 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.976476 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d84559d4-rh27t" event={"ID":"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff","Type":"ContainerStarted","Data":"9f2d3c5308f06ce88581d960f9e9b73481e69597a5e3240b74dccc209dc2a459"} Jul 02 23:11:26.992618 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:26.992564 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" podStartSLOduration=36.842322673 podStartE2EDuration="38.992544815s" totalImagesPullingTime="2.150222142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:26.991128605 +0000 UTC m=+78.986376896" watchObservedRunningTime="2026-07-02 23:11:26.992544815 +0000 UTC m=+78.987793107" Jul 02 23:11:27.092138 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.092097 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb"] Jul 02 23:11:27.095690 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:27.095649 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c64aa82_5448_4089_9fd3_9e89dd921a9b.slice/crio-5d9283c2b9be710c6b59756f6e56dceaff82451578816e0032f2701785cc1c70 WatchSource:0}: Error finding container 5d9283c2b9be710c6b59756f6e56dceaff82451578816e0032f2701785cc1c70: Status 404 returned error can't find the container with id 5d9283c2b9be710c6b59756f6e56dceaff82451578816e0032f2701785cc1c70 Jul 02 23:11:27.779987 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.779930 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-995bfb475-4tww4"] Jul 02 23:11:27.782832 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.782277 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.793046 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.793018 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 23:11:27.793689 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.793623 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-995bfb475-4tww4"] Jul 02 23:11:27.883795 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.883597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-service-ca\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.883795 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.883654 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-serving-cert\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.883795 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.883685 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-config\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.883795 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.883755 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-oauth-serving-cert\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.884139 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.883813 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-trusted-ca-bundle\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.884139 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.883870 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-oauth-config\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.884139 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.883892 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s82t\" (UniqueName: \"kubernetes.io/projected/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-kube-api-access-2s82t\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.983494 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.983458 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" event={"ID":"5c64aa82-5448-4089-9fd3-9e89dd921a9b","Type":"ContainerStarted","Data":"5d9283c2b9be710c6b59756f6e56dceaff82451578816e0032f2701785cc1c70"} Jul 02 23:11:27.985664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.984198 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-serving-cert\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.985664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.984237 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-config\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.985664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.984284 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-oauth-serving-cert\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.985664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.984317 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-trusted-ca-bundle\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.985664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.984350 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-oauth-config\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.985664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.984376 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s82t\" (UniqueName: \"kubernetes.io/projected/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-kube-api-access-2s82t\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.985664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.984431 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-service-ca\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.985664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.985202 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-service-ca\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.986120 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.985748 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-oauth-serving-cert\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.986517 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.986474 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-config\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.986804 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.986781 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-trusted-ca-bundle\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.988481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.988459 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-serving-cert\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.989080 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.989032 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-oauth-config\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:27.995441 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:27.995406 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s82t\" (UniqueName: \"kubernetes.io/projected/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-kube-api-access-2s82t\") pod \"console-995bfb475-4tww4\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:28.095974 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:28.095938 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:34.971128 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:34.971011 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sgl52" Jul 02 23:11:37.120327 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:37.120241 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-995bfb475-4tww4"] Jul 02 23:11:37.124854 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:37.124827 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod713b82f3_e16a_4d3b_b31c_dc74cbd29fa1.slice/crio-e51891b5c150d55238793eaed3e940f44b0696798d4502fbefe17a4756b43ba6 WatchSource:0}: Error finding container e51891b5c150d55238793eaed3e940f44b0696798d4502fbefe17a4756b43ba6: Status 404 returned error can't find the container with id e51891b5c150d55238793eaed3e940f44b0696798d4502fbefe17a4756b43ba6 Jul 02 23:11:37.942146 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:37.942103 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fcb695cd6-p9pt6"] Jul 02 23:11:37.947104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:37.947076 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:11:38.023217 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.023161 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d84559d4-rh27t" event={"ID":"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff","Type":"ContainerStarted","Data":"80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6"} Jul 02 23:11:38.025053 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.025016 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-4z9jb" event={"ID":"5da6a02f-66c5-4af3-b26f-16fb14e7ff2a","Type":"ContainerStarted","Data":"7113755bc497ff996ec61b7c08823929c8c5cf9b88561e26f08c8864c8c79fa2"} Jul 02 23:11:38.025610 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.025585 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-4z9jb" Jul 02 23:11:38.027229 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.027203 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" event={"ID":"5c64aa82-5448-4089-9fd3-9e89dd921a9b","Type":"ContainerStarted","Data":"bfa7b93e5ff218a282d5464443ffe7ec9fce52ef9f83397b9c2a8509400e2d73"} Jul 02 23:11:38.027354 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.027251 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" Jul 02 23:11:38.029060 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.028945 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-995bfb475-4tww4" event={"ID":"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1","Type":"ContainerStarted","Data":"b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a"} Jul 02 23:11:38.029060 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.028984 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-995bfb475-4tww4" event={"ID":"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1","Type":"ContainerStarted","Data":"e51891b5c150d55238793eaed3e940f44b0696798d4502fbefe17a4756b43ba6"} Jul 02 23:11:38.032350 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.032328 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" Jul 02 23:11:38.046795 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.046754 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-4z9jb" Jul 02 23:11:38.096085 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.096046 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:38.096265 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.096115 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:38.102035 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.102006 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:38.107152 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.107094 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58d84559d4-rh27t" podStartSLOduration=2.48311443 podStartE2EDuration="13.107076443s" totalImagesPullingTime="10.623962013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:38.080471176 +0000 UTC m=+90.075719497" watchObservedRunningTime="2026-07-02 23:11:38.107076443 +0000 UTC m=+90.102324714" Jul 02 23:11:38.146741 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.146681 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-4z9jb" podStartSLOduration=2.340921715 podStartE2EDuration="23.146655938s" totalImagesPullingTime="20.805734223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:38.144895242 +0000 UTC m=+90.140143554" watchObservedRunningTime="2026-07-02 23:11:38.146655938 +0000 UTC m=+90.141904231" Jul 02 23:11:38.147234 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.146842 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-n9rnb" podStartSLOduration=2.271813801 podStartE2EDuration="12.146834353s" totalImagesPullingTime="9.875020552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:38.109993224 +0000 UTC m=+90.105241517" watchObservedRunningTime="2026-07-02 23:11:38.146834353 +0000 UTC m=+90.142082644" Jul 02 23:11:38.184464 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.183173 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-995bfb475-4tww4" podStartSLOduration=11.183141481 podStartE2EDuration="11.183141481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:38.182147019 +0000 UTC m=+90.177395316" watchObservedRunningTime="2026-07-02 23:11:38.183141481 +0000 UTC m=+90.178389772" Jul 02 23:11:38.769651 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.769615 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r"] Jul 02 23:11:38.802140 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.802108 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r"] Jul 02 23:11:38.802323 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.802267 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.807553 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.807518 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 02 23:11:38.807714 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.807572 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 02 23:11:38.808996 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.808955 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 23:11:38.809127 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.809026 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jgc55\"" Jul 02 23:11:38.875557 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.875524 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/600b96fe-8a50-40ee-94c0-825e04b576ce-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.875743 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.875573 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/600b96fe-8a50-40ee-94c0-825e04b576ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.875743 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.875616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/600b96fe-8a50-40ee-94c0-825e04b576ce-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.875743 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.875662 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psfd4\" (UniqueName: \"kubernetes.io/projected/600b96fe-8a50-40ee-94c0-825e04b576ce-kube-api-access-psfd4\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.976492 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.976453 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/600b96fe-8a50-40ee-94c0-825e04b576ce-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.976689 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.976515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psfd4\" (UniqueName: \"kubernetes.io/projected/600b96fe-8a50-40ee-94c0-825e04b576ce-kube-api-access-psfd4\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.976689 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.976596 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/600b96fe-8a50-40ee-94c0-825e04b576ce-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.976689 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.976637 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/600b96fe-8a50-40ee-94c0-825e04b576ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.977702 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.977670 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/600b96fe-8a50-40ee-94c0-825e04b576ce-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.979655 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.979627 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/600b96fe-8a50-40ee-94c0-825e04b576ce-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.979784 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.979638 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/600b96fe-8a50-40ee-94c0-825e04b576ce-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:38.987250 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:38.987223 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psfd4\" (UniqueName: \"kubernetes.io/projected/600b96fe-8a50-40ee-94c0-825e04b576ce-kube-api-access-psfd4\") pod \"prometheus-operator-567f7b8b7c-q4z6r\" (UID: \"600b96fe-8a50-40ee-94c0-825e04b576ce\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:39.038488 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:39.038399 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:11:39.102073 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:39.102036 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d84559d4-rh27t"] Jul 02 23:11:39.114332 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:39.114302 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" Jul 02 23:11:39.291130 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:39.291049 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r"] Jul 02 23:11:39.295328 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:39.294440 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600b96fe_8a50_40ee_94c0_825e04b576ce.slice/crio-0941eea7200613a4e282770040268fe10442bad7b12b767e0ffd99349bcb77c5 WatchSource:0}: Error finding container 0941eea7200613a4e282770040268fe10442bad7b12b767e0ffd99349bcb77c5: Status 404 returned error can't find the container with id 0941eea7200613a4e282770040268fe10442bad7b12b767e0ffd99349bcb77c5 Jul 02 23:11:40.041357 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:40.041316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" event={"ID":"600b96fe-8a50-40ee-94c0-825e04b576ce","Type":"ContainerStarted","Data":"0941eea7200613a4e282770040268fe10442bad7b12b767e0ffd99349bcb77c5"} Jul 02 23:11:42.052457 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:42.052354 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" event={"ID":"600b96fe-8a50-40ee-94c0-825e04b576ce","Type":"ContainerStarted","Data":"d02c717a4d3ac391ef6efa0468fa72bb777a725ea911a3abe8b1a2ceaabad80a"} Jul 02 23:11:42.052457 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:42.052423 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" event={"ID":"600b96fe-8a50-40ee-94c0-825e04b576ce","Type":"ContainerStarted","Data":"094c7d633571ef9cb262a87dbb136a098f6d4fb4805740c870a78422845f8d16"} Jul 02 23:11:42.081066 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:42.081009 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4z6r" podStartSLOduration=1.999724697 podStartE2EDuration="4.080987751s" totalImagesPullingTime="2.081263054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:42.078583284 +0000 UTC m=+94.073831600" watchObservedRunningTime="2026-07-02 23:11:42.080987751 +0000 UTC m=+94.076236042" Jul 02 23:11:44.191550 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.191508 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk"] Jul 02 23:11:44.223483 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.223444 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lvgbp"] Jul 02 23:11:44.223668 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.223600 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.227075 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.227049 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 02 23:11:44.228572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.228436 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 02 23:11:44.228572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.228448 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 02 23:11:44.228572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.228480 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w8mp4\"" Jul 02 23:11:44.243107 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.243077 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk"] Jul 02 23:11:44.243267 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.243226 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.247058 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.247033 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 23:11:44.247195 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.247083 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 23:11:44.247195 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.247158 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bdzrc\"" Jul 02 23:11:44.247478 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.247445 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 23:11:44.325150 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325116 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.325150 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325157 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-sys\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.325422 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325251 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-root\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.325422 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325278 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gls9p\" (UniqueName: \"kubernetes.io/projected/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-api-access-gls9p\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.325422 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325345 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjrq7\" (UniqueName: \"kubernetes.io/projected/9e544b8e-e401-4b88-ab70-30981b96a3dc-kube-api-access-hjrq7\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.325628 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325435 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.325628 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325470 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.325628 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325505 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.325628 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325576 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.325815 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325662 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-tls\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.325815 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.325815 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.325815 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325732 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-wtmp\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.325815 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325767 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-textfile\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.325815 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.325809 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e544b8e-e401-4b88-ab70-30981b96a3dc-metrics-client-ca\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.427113 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.427070 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.427331 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.427134 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.428337 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428259 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.428541 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428512 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-tls\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.428661 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428566 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.428661 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428600 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.428661 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428632 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-wtmp\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.428809 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428664 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-textfile\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.428809 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e544b8e-e401-4b88-ab70-30981b96a3dc-metrics-client-ca\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.428809 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428748 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.428809 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428788 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-sys\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.428986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428833 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-root\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.428986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428866 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gls9p\" (UniqueName: \"kubernetes.io/projected/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-api-access-gls9p\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.428986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428908 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjrq7\" (UniqueName: \"kubernetes.io/projected/9e544b8e-e401-4b88-ab70-30981b96a3dc-kube-api-access-hjrq7\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.428986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.428980 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.429163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.429017 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.430231 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.429466 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.430231 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:44.429595 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 23:11:44.430231 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:11:44.429658 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-tls podName:9e544b8e-e401-4b88-ab70-30981b96a3dc nodeName:}" failed. No retries permitted until 2026-07-02 23:11:44.929639743 +0000 UTC m=+96.924888012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-tls") pod "node-exporter-lvgbp" (UID: "9e544b8e-e401-4b88-ab70-30981b96a3dc") : secret "node-exporter-tls" not found Jul 02 23:11:44.430418 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.430297 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-sys\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.430695 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.430673 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-textfile\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.430941 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.430913 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.434096 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.431203 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-wtmp\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.434096 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.431458 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e544b8e-e401-4b88-ab70-30981b96a3dc-metrics-client-ca\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.434096 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.431575 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e544b8e-e401-4b88-ab70-30981b96a3dc-root\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.434096 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.432131 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.434378 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.434278 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.434378 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.434330 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.437171 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.437150 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.443004 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.442944 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gls9p\" (UniqueName: \"kubernetes.io/projected/ad411b0b-0075-4295-9a3c-ff0e68d8da8f-kube-api-access-gls9p\") pod \"kube-state-metrics-7c9dbb8b7d-tcjmk\" (UID: \"ad411b0b-0075-4295-9a3c-ff0e68d8da8f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.443104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.443045 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjrq7\" (UniqueName: \"kubernetes.io/projected/9e544b8e-e401-4b88-ab70-30981b96a3dc-kube-api-access-hjrq7\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.536349 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.536309 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" Jul 02 23:11:44.736288 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.736201 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk"] Jul 02 23:11:44.739833 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:44.739802 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad411b0b_0075_4295_9a3c_ff0e68d8da8f.slice/crio-0802ad130936a618b7b4924464643c3971c9d86039a27daf2a250d85cac27bc7 WatchSource:0}: Error finding container 0802ad130936a618b7b4924464643c3971c9d86039a27daf2a250d85cac27bc7: Status 404 returned error can't find the container with id 0802ad130936a618b7b4924464643c3971c9d86039a27daf2a250d85cac27bc7 Jul 02 23:11:44.933967 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.933930 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-tls\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:44.936812 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:44.936783 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e544b8e-e401-4b88-ab70-30981b96a3dc-node-exporter-tls\") pod \"node-exporter-lvgbp\" (UID: \"9e544b8e-e401-4b88-ab70-30981b96a3dc\") " pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:45.068388 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:45.068300 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" event={"ID":"ad411b0b-0075-4295-9a3c-ff0e68d8da8f","Type":"ContainerStarted","Data":"0802ad130936a618b7b4924464643c3971c9d86039a27daf2a250d85cac27bc7"} Jul 02 23:11:45.153834 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:45.153796 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lvgbp" Jul 02 23:11:45.199329 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:45.199294 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e544b8e_e401_4b88_ab70_30981b96a3dc.slice/crio-212bb1f6746c9a474d2ac910981897346f74713aa2c1e32f05966784e2bee42c WatchSource:0}: Error finding container 212bb1f6746c9a474d2ac910981897346f74713aa2c1e32f05966784e2bee42c: Status 404 returned error can't find the container with id 212bb1f6746c9a474d2ac910981897346f74713aa2c1e32f05966784e2bee42c Jul 02 23:11:46.073522 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:46.073479 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lvgbp" event={"ID":"9e544b8e-e401-4b88-ab70-30981b96a3dc","Type":"ContainerStarted","Data":"212bb1f6746c9a474d2ac910981897346f74713aa2c1e32f05966784e2bee42c"} Jul 02 23:11:46.134893 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:46.134816 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:11:48.083330 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.083291 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" event={"ID":"ad411b0b-0075-4295-9a3c-ff0e68d8da8f","Type":"ContainerStarted","Data":"2295e676637bd51f9612734d3982220a2a77fbb3ba707a3fa1f5bc6e358b8a65"} Jul 02 23:11:48.083798 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.083360 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" event={"ID":"ad411b0b-0075-4295-9a3c-ff0e68d8da8f","Type":"ContainerStarted","Data":"ed6b4104c5e958cb31e47939c5870ffd0db045ec1492b9260f62a102572c62b3"} Jul 02 23:11:48.083798 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.083388 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" event={"ID":"ad411b0b-0075-4295-9a3c-ff0e68d8da8f","Type":"ContainerStarted","Data":"41be97a037bc6fe62f3aebb85047d68a7c9da8dc1cdd9b83450e5b9b8f5551c7"} Jul 02 23:11:48.085076 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.085047 2570 generic.go:356] "Generic (PLEG): container finished" podID="9e544b8e-e401-4b88-ab70-30981b96a3dc" containerID="36647bc20f9e79bdc6467e40288d27ca274955f28cc71a6b474af7fd3cb20d1c" exitCode=0 Jul 02 23:11:48.085313 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.085293 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lvgbp" event={"ID":"9e544b8e-e401-4b88-ab70-30981b96a3dc","Type":"ContainerDied","Data":"36647bc20f9e79bdc6467e40288d27ca274955f28cc71a6b474af7fd3cb20d1c"} Jul 02 23:11:48.127519 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.127475 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-tcjmk" podStartSLOduration=1.777398434 podStartE2EDuration="4.12745926s" totalImagesPullingTime="2.350060826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:48.126073963 +0000 UTC m=+100.121322304" watchObservedRunningTime="2026-07-02 23:11:48.12745926 +0000 UTC m=+100.122707587" Jul 02 23:11:48.920102 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.920072 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76ccc458c9-6cnkr"] Jul 02 23:11:48.937299 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.937256 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76ccc458c9-6cnkr"] Jul 02 23:11:48.937485 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.937393 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:48.968807 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.968781 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-service-ca\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:48.968938 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.968821 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-trusted-ca-bundle\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:48.968938 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.968845 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-oauth-config\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:48.968938 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.968862 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-config\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:48.968938 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.968880 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-oauth-serving-cert\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:48.969082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.968939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crnnx\" (UniqueName: \"kubernetes.io/projected/0d84b754-9cac-43f5-8b7b-f75120c70cf6-kube-api-access-crnnx\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:48.969082 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:48.968957 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-serving-cert\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.069904 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.069867 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-oauth-config\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.069904 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.069903 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-config\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.070150 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.069921 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-oauth-serving-cert\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.070150 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.070138 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crnnx\" (UniqueName: \"kubernetes.io/projected/0d84b754-9cac-43f5-8b7b-f75120c70cf6-kube-api-access-crnnx\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.070302 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.070160 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-serving-cert\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.070302 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.070211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-service-ca\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.070302 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.070254 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-trusted-ca-bundle\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.070820 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.070789 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-oauth-serving-cert\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.070941 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.070799 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-config\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.071053 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.071028 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-service-ca\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.071116 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.071063 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-trusted-ca-bundle\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.072436 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.072411 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-oauth-config\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.072658 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.072641 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-serving-cert\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.078914 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.078893 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crnnx\" (UniqueName: \"kubernetes.io/projected/0d84b754-9cac-43f5-8b7b-f75120c70cf6-kube-api-access-crnnx\") pod \"console-76ccc458c9-6cnkr\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.091138 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.091107 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lvgbp" event={"ID":"9e544b8e-e401-4b88-ab70-30981b96a3dc","Type":"ContainerStarted","Data":"c905b920eb78700b45c5b47c3e4d5f8b376d306efb5ad1c400627d6825797f61"} Jul 02 23:11:49.091431 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.091174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lvgbp" event={"ID":"9e544b8e-e401-4b88-ab70-30981b96a3dc","Type":"ContainerStarted","Data":"de6a6da6854123d8cd2ae802193df810f5b924a0c61278a6aab292a46a849a77"} Jul 02 23:11:49.115599 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.115549 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lvgbp" podStartSLOduration=3.106390485 podStartE2EDuration="5.115532655s" totalImagesPullingTime="1.89249s" totalInitContainerRuntime="116.65217ms" isStatefulPod=true podCreationTimestamp="2026-07-02 23:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:49.114255945 +0000 UTC m=+101.109504254" watchObservedRunningTime="2026-07-02 23:11:49.115532655 +0000 UTC m=+101.110780946" Jul 02 23:11:49.247663 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.247569 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:11:49.407615 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:49.407582 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76ccc458c9-6cnkr"] Jul 02 23:11:49.410719 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:49.410692 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d84b754_9cac_43f5_8b7b_f75120c70cf6.slice/crio-8436ca4cb91b73e872f173b27a4b9ff60c65f83802c90469a9b8931cda0a0559 WatchSource:0}: Error finding container 8436ca4cb91b73e872f173b27a4b9ff60c65f83802c90469a9b8931cda0a0559: Status 404 returned error can't find the container with id 8436ca4cb91b73e872f173b27a4b9ff60c65f83802c90469a9b8931cda0a0559 Jul 02 23:11:50.101107 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:50.101069 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76ccc458c9-6cnkr" event={"ID":"0d84b754-9cac-43f5-8b7b-f75120c70cf6","Type":"ContainerStarted","Data":"b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f"} Jul 02 23:11:50.101597 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:50.101124 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76ccc458c9-6cnkr" event={"ID":"0d84b754-9cac-43f5-8b7b-f75120c70cf6","Type":"ContainerStarted","Data":"8436ca4cb91b73e872f173b27a4b9ff60c65f83802c90469a9b8931cda0a0559"} Jul 02 23:11:50.120756 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:50.120702 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76ccc458c9-6cnkr" podStartSLOduration=2.120683641 podStartE2EDuration="2.120683641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:50.119537059 +0000 UTC m=+102.114785369" watchObservedRunningTime="2026-07-02 23:11:50.120683641 +0000 UTC m=+102.115931985" Jul 02 23:11:52.820240 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:52.820207 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76ccc458c9-6cnkr"] Jul 02 23:11:52.862524 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:52.862485 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6585bb9d54-44plb"] Jul 02 23:11:52.901571 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:52.901529 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6585bb9d54-44plb"] Jul 02 23:11:52.901730 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:52.901585 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.004905 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.004868 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-config\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.005089 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.004915 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-serving-cert\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.005089 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.004977 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-trusted-ca-bundle\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.005089 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.005026 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-service-ca\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.005089 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.005048 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-oauth-serving-cert\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.005089 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.005073 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp6jt\" (UniqueName: \"kubernetes.io/projected/93e22820-d5f6-42dd-893a-7bdd755d5bc9-kube-api-access-zp6jt\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.005281 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.005095 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-oauth-config\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.106156 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.106120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-serving-cert\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.106156 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.106161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-trusted-ca-bundle\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.106549 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.106221 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-service-ca\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.106549 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.106248 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-oauth-serving-cert\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.106549 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.106277 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp6jt\" (UniqueName: \"kubernetes.io/projected/93e22820-d5f6-42dd-893a-7bdd755d5bc9-kube-api-access-zp6jt\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.106549 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.106299 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-oauth-config\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.106549 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.106346 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-config\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.107174 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.107149 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-config\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.107174 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.107169 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-service-ca\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.107334 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.107171 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-oauth-serving-cert\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.107425 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.107407 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-trusted-ca-bundle\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.109260 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.109237 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-serving-cert\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.109369 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.109314 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-oauth-config\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.119687 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.119658 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp6jt\" (UniqueName: \"kubernetes.io/projected/93e22820-d5f6-42dd-893a-7bdd755d5bc9-kube-api-access-zp6jt\") pod \"console-6585bb9d54-44plb\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.210663 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.210615 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:11:53.379745 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:53.379723 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6585bb9d54-44plb"] Jul 02 23:11:53.381126 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:11:53.381087 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e22820_d5f6_42dd_893a_7bdd755d5bc9.slice/crio-e3ae5ba5e918d8e1c97f17e9704fbd78f173e9bae0854d0732fcc36fdf61486f WatchSource:0}: Error finding container e3ae5ba5e918d8e1c97f17e9704fbd78f173e9bae0854d0732fcc36fdf61486f: Status 404 returned error can't find the container with id e3ae5ba5e918d8e1c97f17e9704fbd78f173e9bae0854d0732fcc36fdf61486f Jul 02 23:11:54.118618 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:54.118579 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6585bb9d54-44plb" event={"ID":"93e22820-d5f6-42dd-893a-7bdd755d5bc9","Type":"ContainerStarted","Data":"68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646"} Jul 02 23:11:54.119064 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:54.118659 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6585bb9d54-44plb" event={"ID":"93e22820-d5f6-42dd-893a-7bdd755d5bc9","Type":"ContainerStarted","Data":"e3ae5ba5e918d8e1c97f17e9704fbd78f173e9bae0854d0732fcc36fdf61486f"} Jul 02 23:11:54.139458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:54.139403 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6585bb9d54-44plb" podStartSLOduration=2.139386772 podStartE2EDuration="2.139386772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:54.138236995 +0000 UTC m=+106.133485286" watchObservedRunningTime="2026-07-02 23:11:54.139386772 +0000 UTC m=+106.134635144" Jul 02 23:11:55.971919 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:55.971891 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zlls4" Jul 02 23:11:59.248642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:11:59.248603 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:12:02.967447 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:02.967392 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" podUID="93c5d449-8ad5-45ef-b99b-afa26ff912d6" containerName="registry" containerID="cri-o://c0bed6161870776de78e61ed8549a5fa7f1e325f6808b925d917069c1f8bd9ff" gracePeriod=30 Jul 02 23:12:03.158132 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.158099 2570 generic.go:356] "Generic (PLEG): container finished" podID="93c5d449-8ad5-45ef-b99b-afa26ff912d6" containerID="c0bed6161870776de78e61ed8549a5fa7f1e325f6808b925d917069c1f8bd9ff" exitCode=0 Jul 02 23:12:03.158294 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.158149 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" event={"ID":"93c5d449-8ad5-45ef-b99b-afa26ff912d6","Type":"ContainerDied","Data":"c0bed6161870776de78e61ed8549a5fa7f1e325f6808b925d917069c1f8bd9ff"} Jul 02 23:12:03.211620 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.211590 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:12:03.217002 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.212084 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:12:03.221405 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.221375 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:12:03.250423 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.250383 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:12:03.395245 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395205 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jldkz\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-kube-api-access-jldkz\") pod \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " Jul 02 23:12:03.395432 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395256 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-installation-pull-secrets\") pod \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " Jul 02 23:12:03.395432 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395291 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-bound-sa-token\") pod \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " Jul 02 23:12:03.395432 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395323 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-image-registry-private-configuration\") pod \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " Jul 02 23:12:03.395432 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395411 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-trusted-ca\") pod \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " Jul 02 23:12:03.395667 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395450 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") pod \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " Jul 02 23:12:03.395667 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395490 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-certificates\") pod \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " Jul 02 23:12:03.395667 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395520 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93c5d449-8ad5-45ef-b99b-afa26ff912d6-ca-trust-extracted\") pod \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\" (UID: \"93c5d449-8ad5-45ef-b99b-afa26ff912d6\") " Jul 02 23:12:03.396010 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.395980 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "93c5d449-8ad5-45ef-b99b-afa26ff912d6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:03.396176 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.396038 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "93c5d449-8ad5-45ef-b99b-afa26ff912d6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:03.397988 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.397948 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-kube-api-access-jldkz" (OuterVolumeSpecName: "kube-api-access-jldkz") pod "93c5d449-8ad5-45ef-b99b-afa26ff912d6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6"). InnerVolumeSpecName "kube-api-access-jldkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:03.398315 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.398289 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "93c5d449-8ad5-45ef-b99b-afa26ff912d6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:03.398450 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.398427 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "93c5d449-8ad5-45ef-b99b-afa26ff912d6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:03.398450 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.398437 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "93c5d449-8ad5-45ef-b99b-afa26ff912d6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:03.398550 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.398469 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "93c5d449-8ad5-45ef-b99b-afa26ff912d6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:03.404519 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.404490 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c5d449-8ad5-45ef-b99b-afa26ff912d6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "93c5d449-8ad5-45ef-b99b-afa26ff912d6" (UID: "93c5d449-8ad5-45ef-b99b-afa26ff912d6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:12:03.496977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.496891 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-certificates\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:03.496977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.496926 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93c5d449-8ad5-45ef-b99b-afa26ff912d6-ca-trust-extracted\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:03.496977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.496934 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jldkz\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-kube-api-access-jldkz\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:03.496977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.496943 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-installation-pull-secrets\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:03.496977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.496951 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-bound-sa-token\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:03.496977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.496961 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93c5d449-8ad5-45ef-b99b-afa26ff912d6-image-registry-private-configuration\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:03.496977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.496969 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93c5d449-8ad5-45ef-b99b-afa26ff912d6-trusted-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:03.496977 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:03.496977 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93c5d449-8ad5-45ef-b99b-afa26ff912d6-registry-tls\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:04.163500 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.163469 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" Jul 02 23:12:04.163934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.163502 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcb695cd6-p9pt6" event={"ID":"93c5d449-8ad5-45ef-b99b-afa26ff912d6","Type":"ContainerDied","Data":"9117a7c737d25cf21767fab9c6bdff7a225cabc3cfece268f7e1899b79b5054d"} Jul 02 23:12:04.163934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.163561 2570 scope.go:117] "RemoveContainer" containerID="c0bed6161870776de78e61ed8549a5fa7f1e325f6808b925d917069c1f8bd9ff" Jul 02 23:12:04.165113 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.165092 2570 generic.go:356] "Generic (PLEG): container finished" podID="2384cb4b-da67-4522-8ca7-d721551a31bb" containerID="e511137be9bf3d4a47e311b198d11b5524857a6fa2a142532e7aaa0694ee1460" exitCode=0 Jul 02 23:12:04.165235 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.165215 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" event={"ID":"2384cb4b-da67-4522-8ca7-d721551a31bb","Type":"ContainerDied","Data":"e511137be9bf3d4a47e311b198d11b5524857a6fa2a142532e7aaa0694ee1460"} Jul 02 23:12:04.165666 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.165649 2570 scope.go:117] "RemoveContainer" containerID="e511137be9bf3d4a47e311b198d11b5524857a6fa2a142532e7aaa0694ee1460" Jul 02 23:12:04.170411 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.170391 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:12:04.213237 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.211780 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fcb695cd6-p9pt6"] Jul 02 23:12:04.213237 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.211838 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7fcb695cd6-p9pt6"] Jul 02 23:12:04.270084 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.270046 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-995bfb475-4tww4"] Jul 02 23:12:04.593038 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:04.593002 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93c5d449-8ad5-45ef-b99b-afa26ff912d6" path="/var/lib/kubelet/pods/93c5d449-8ad5-45ef-b99b-afa26ff912d6/volumes" Jul 02 23:12:05.071423 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.071376 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-58d84559d4-rh27t" podUID="dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" containerName="console" containerID="cri-o://80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6" gracePeriod=15 Jul 02 23:12:05.171962 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.171928 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-kb5t6" event={"ID":"2384cb4b-da67-4522-8ca7-d721551a31bb","Type":"ContainerStarted","Data":"100798ce05b9f4cc746cc38592f4bd77a138b3f260b6f39b44500ea62586c969"} Jul 02 23:12:05.348821 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.348746 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:12:05.417073 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417041 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-service-ca\") pod \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " Jul 02 23:12:05.417073 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417086 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-config\") pod \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " Jul 02 23:12:05.417323 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417123 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-oauth-config\") pod \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " Jul 02 23:12:05.417323 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417192 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-oauth-serving-cert\") pod \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " Jul 02 23:12:05.417323 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417215 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-serving-cert\") pod \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " Jul 02 23:12:05.417323 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417282 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5m9j\" (UniqueName: \"kubernetes.io/projected/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-kube-api-access-b5m9j\") pod \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\" (UID: \"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff\") " Jul 02 23:12:05.417583 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417543 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-service-ca" (OuterVolumeSpecName: "service-ca") pod "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" (UID: "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:05.417706 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417594 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-config" (OuterVolumeSpecName: "console-config") pod "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" (UID: "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:05.417706 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.417599 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" (UID: "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:05.419491 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.419470 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" (UID: "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:05.419619 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.419600 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" (UID: "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:05.419727 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.419711 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-kube-api-access-b5m9j" (OuterVolumeSpecName: "kube-api-access-b5m9j") pod "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" (UID: "dc7ca2b1-5f0a-4016-8661-40f6c87f54ff"). InnerVolumeSpecName "kube-api-access-b5m9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:05.518353 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.518309 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-oauth-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:05.518353 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.518344 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:05.518353 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.518354 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5m9j\" (UniqueName: \"kubernetes.io/projected/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-kube-api-access-b5m9j\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:05.518593 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.518366 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:05.518593 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.518375 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:05.518593 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:05.518383 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff-console-oauth-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:06.176703 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.176659 2570 generic.go:356] "Generic (PLEG): container finished" podID="dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" containerID="80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6" exitCode=0 Jul 02 23:12:06.177106 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.176727 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58d84559d4-rh27t" Jul 02 23:12:06.177106 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.176742 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d84559d4-rh27t" event={"ID":"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff","Type":"ContainerDied","Data":"80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6"} Jul 02 23:12:06.177106 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.176824 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58d84559d4-rh27t" event={"ID":"dc7ca2b1-5f0a-4016-8661-40f6c87f54ff","Type":"ContainerDied","Data":"9f2d3c5308f06ce88581d960f9e9b73481e69597a5e3240b74dccc209dc2a459"} Jul 02 23:12:06.177106 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.176845 2570 scope.go:117] "RemoveContainer" containerID="80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6" Jul 02 23:12:06.185306 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.185282 2570 scope.go:117] "RemoveContainer" containerID="80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6" Jul 02 23:12:06.185606 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:12:06.185580 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6\": container with ID starting with 80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6 not found: ID does not exist" containerID="80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6" Jul 02 23:12:06.185691 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.185611 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6"} err="failed to get container status \"80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6\": rpc error: code = NotFound desc = could not find container \"80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6\": container with ID starting with 80b12a76e8cbd6708e86132db1cb7e7762ecd746da7c36c41474c30ac0b2ade6 not found: ID does not exist" Jul 02 23:12:06.198259 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.198234 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58d84559d4-rh27t"] Jul 02 23:12:06.204536 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.204511 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58d84559d4-rh27t"] Jul 02 23:12:06.592043 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:06.592008 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" path="/var/lib/kubelet/pods/dc7ca2b1-5f0a-4016-8661-40f6c87f54ff/volumes" Jul 02 23:12:17.842232 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:17.842151 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76ccc458c9-6cnkr" podUID="0d84b754-9cac-43f5-8b7b-f75120c70cf6" containerName="console" containerID="cri-o://b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f" gracePeriod=15 Jul 02 23:12:18.113740 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.113657 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:12:18.131969 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.131944 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-config\") pod \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " Jul 02 23:12:18.132099 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.131986 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-service-ca\") pod \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " Jul 02 23:12:18.132099 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132002 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-oauth-config\") pod \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " Jul 02 23:12:18.132099 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132032 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crnnx\" (UniqueName: \"kubernetes.io/projected/0d84b754-9cac-43f5-8b7b-f75120c70cf6-kube-api-access-crnnx\") pod \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " Jul 02 23:12:18.132296 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132144 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-serving-cert\") pod \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " Jul 02 23:12:18.132296 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132209 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-oauth-serving-cert\") pod \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " Jul 02 23:12:18.132296 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132250 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-trusted-ca-bundle\") pod \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\" (UID: \"0d84b754-9cac-43f5-8b7b-f75120c70cf6\") " Jul 02 23:12:18.132456 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132361 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-config" (OuterVolumeSpecName: "console-config") pod "0d84b754-9cac-43f5-8b7b-f75120c70cf6" (UID: "0d84b754-9cac-43f5-8b7b-f75120c70cf6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:18.132511 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132502 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:18.132566 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132522 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-service-ca" (OuterVolumeSpecName: "service-ca") pod "0d84b754-9cac-43f5-8b7b-f75120c70cf6" (UID: "0d84b754-9cac-43f5-8b7b-f75120c70cf6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:18.132604 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132577 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0d84b754-9cac-43f5-8b7b-f75120c70cf6" (UID: "0d84b754-9cac-43f5-8b7b-f75120c70cf6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:18.132653 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.132632 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0d84b754-9cac-43f5-8b7b-f75120c70cf6" (UID: "0d84b754-9cac-43f5-8b7b-f75120c70cf6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:18.134438 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.134414 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d84b754-9cac-43f5-8b7b-f75120c70cf6-kube-api-access-crnnx" (OuterVolumeSpecName: "kube-api-access-crnnx") pod "0d84b754-9cac-43f5-8b7b-f75120c70cf6" (UID: "0d84b754-9cac-43f5-8b7b-f75120c70cf6"). InnerVolumeSpecName "kube-api-access-crnnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:18.134722 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.134703 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0d84b754-9cac-43f5-8b7b-f75120c70cf6" (UID: "0d84b754-9cac-43f5-8b7b-f75120c70cf6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:18.134789 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.134723 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0d84b754-9cac-43f5-8b7b-f75120c70cf6" (UID: "0d84b754-9cac-43f5-8b7b-f75120c70cf6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:18.232495 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232461 2570 generic.go:356] "Generic (PLEG): container finished" podID="0d84b754-9cac-43f5-8b7b-f75120c70cf6" containerID="b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f" exitCode=0 Jul 02 23:12:18.232676 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232506 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76ccc458c9-6cnkr" Jul 02 23:12:18.232676 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232537 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76ccc458c9-6cnkr" event={"ID":"0d84b754-9cac-43f5-8b7b-f75120c70cf6","Type":"ContainerDied","Data":"b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f"} Jul 02 23:12:18.232676 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232589 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76ccc458c9-6cnkr" event={"ID":"0d84b754-9cac-43f5-8b7b-f75120c70cf6","Type":"ContainerDied","Data":"8436ca4cb91b73e872f173b27a4b9ff60c65f83802c90469a9b8931cda0a0559"} Jul 02 23:12:18.232676 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232610 2570 scope.go:117] "RemoveContainer" containerID="b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f" Jul 02 23:12:18.232885 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232863 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:18.232940 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232886 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-oauth-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:18.232940 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232901 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crnnx\" (UniqueName: \"kubernetes.io/projected/0d84b754-9cac-43f5-8b7b-f75120c70cf6-kube-api-access-crnnx\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:18.232940 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232915 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0d84b754-9cac-43f5-8b7b-f75120c70cf6-console-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:18.232940 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232929 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-oauth-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:18.233098 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.232944 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d84b754-9cac-43f5-8b7b-f75120c70cf6-trusted-ca-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:18.245053 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.245036 2570 scope.go:117] "RemoveContainer" containerID="b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f" Jul 02 23:12:18.245338 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:12:18.245318 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f\": container with ID starting with b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f not found: ID does not exist" containerID="b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f" Jul 02 23:12:18.245399 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.245346 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f"} err="failed to get container status \"b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f\": rpc error: code = NotFound desc = could not find container \"b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f\": container with ID starting with b70c7878a213e1ae66ee1094dd0111d3d6726c25e9f6e4f56594455419633c4f not found: ID does not exist" Jul 02 23:12:18.256600 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.256571 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76ccc458c9-6cnkr"] Jul 02 23:12:18.260402 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.260378 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76ccc458c9-6cnkr"] Jul 02 23:12:18.592013 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:18.591982 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d84b754-9cac-43f5-8b7b-f75120c70cf6" path="/var/lib/kubelet/pods/0d84b754-9cac-43f5-8b7b-f75120c70cf6/volumes" Jul 02 23:12:19.238238 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:19.238139 2570 generic.go:356] "Generic (PLEG): container finished" podID="2ce22038-a78d-48ec-824c-1e1ff97400d2" containerID="ad46eea139bf108f49aeefab5af7809bc7fced8b1f9c93abbab7c42c32fcf69e" exitCode=0 Jul 02 23:12:19.238238 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:19.238214 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" event={"ID":"2ce22038-a78d-48ec-824c-1e1ff97400d2","Type":"ContainerDied","Data":"ad46eea139bf108f49aeefab5af7809bc7fced8b1f9c93abbab7c42c32fcf69e"} Jul 02 23:12:19.238768 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:19.238508 2570 scope.go:117] "RemoveContainer" containerID="ad46eea139bf108f49aeefab5af7809bc7fced8b1f9c93abbab7c42c32fcf69e" Jul 02 23:12:20.243601 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:20.243569 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-t849h" event={"ID":"2ce22038-a78d-48ec-824c-1e1ff97400d2","Type":"ContainerStarted","Data":"cb81348573f308f4fc2265a7ebfaf7f71cd3656dfced1957b1e2872e236e93cd"} Jul 02 23:12:29.291551 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.291485 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-995bfb475-4tww4" podUID="713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" containerName="console" containerID="cri-o://b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a" gracePeriod=15 Jul 02 23:12:29.554268 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.554199 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:12:29.612318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612283 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-serving-cert\") pod \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " Jul 02 23:12:29.612504 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612368 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-oauth-config\") pod \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " Jul 02 23:12:29.612504 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612404 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-service-ca\") pod \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " Jul 02 23:12:29.612504 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612429 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-oauth-serving-cert\") pod \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " Jul 02 23:12:29.612504 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612456 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-trusted-ca-bundle\") pod \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " Jul 02 23:12:29.612504 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612490 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-config\") pod \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " Jul 02 23:12:29.612768 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612532 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2s82t\" (UniqueName: \"kubernetes.io/projected/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-kube-api-access-2s82t\") pod \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\" (UID: \"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1\") " Jul 02 23:12:29.613235 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612862 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-service-ca" (OuterVolumeSpecName: "service-ca") pod "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" (UID: "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:29.613235 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.612932 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" (UID: "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:29.613235 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.613170 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-config" (OuterVolumeSpecName: "console-config") pod "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" (UID: "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:29.613474 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.613253 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" (UID: "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:12:29.614690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.614668 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" (UID: "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:29.614798 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.614770 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-kube-api-access-2s82t" (OuterVolumeSpecName: "kube-api-access-2s82t") pod "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" (UID: "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1"). InnerVolumeSpecName "kube-api-access-2s82t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:12:29.614922 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.614904 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" (UID: "713b82f3-e16a-4d3b-b31c-dc74cbd29fa1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:12:29.713645 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.713596 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:29.713645 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.713644 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-oauth-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:29.713853 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.713659 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-trusted-ca-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:29.713853 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.713674 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:29.713853 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.713690 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s82t\" (UniqueName: \"kubernetes.io/projected/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-kube-api-access-2s82t\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:29.713853 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.713704 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:29.713853 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:29.713719 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1-console-oauth-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:12:30.280999 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.280957 2570 generic.go:356] "Generic (PLEG): container finished" podID="713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" containerID="b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a" exitCode=0 Jul 02 23:12:30.281167 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.281010 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-995bfb475-4tww4" Jul 02 23:12:30.281167 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.281047 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-995bfb475-4tww4" event={"ID":"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1","Type":"ContainerDied","Data":"b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a"} Jul 02 23:12:30.281167 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.281104 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-995bfb475-4tww4" event={"ID":"713b82f3-e16a-4d3b-b31c-dc74cbd29fa1","Type":"ContainerDied","Data":"e51891b5c150d55238793eaed3e940f44b0696798d4502fbefe17a4756b43ba6"} Jul 02 23:12:30.281167 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.281128 2570 scope.go:117] "RemoveContainer" containerID="b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a" Jul 02 23:12:30.290496 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.290479 2570 scope.go:117] "RemoveContainer" containerID="b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a" Jul 02 23:12:30.290770 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:12:30.290749 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a\": container with ID starting with b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a not found: ID does not exist" containerID="b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a" Jul 02 23:12:30.290819 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.290779 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a"} err="failed to get container status \"b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a\": rpc error: code = NotFound desc = could not find container \"b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a\": container with ID starting with b25d9067aede8c403da06e50f513eeed9585b13d3b440cecaa72bb058a55038a not found: ID does not exist" Jul 02 23:12:30.304978 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.304953 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-995bfb475-4tww4"] Jul 02 23:12:30.309379 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.309360 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-995bfb475-4tww4"] Jul 02 23:12:30.591975 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:12:30.591942 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" path="/var/lib/kubelet/pods/713b82f3-e16a-4d3b-b31c-dc74cbd29fa1/volumes" Jul 02 23:13:04.405553 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:04.405528 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:13:04.405955 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:04.405568 2570 generic.go:356] "Generic (PLEG): container finished" podID="b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9" containerID="1c03411cc20f23e9fe225e6bf34cba1e632c218badcbadd5e3ff3d34ec890fb6" exitCode=2 Jul 02 23:13:04.405955 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:04.405597 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" event={"ID":"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9","Type":"ContainerDied","Data":"1c03411cc20f23e9fe225e6bf34cba1e632c218badcbadd5e3ff3d34ec890fb6"} Jul 02 23:13:04.405955 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:04.405879 2570 scope.go:117] "RemoveContainer" containerID="1c03411cc20f23e9fe225e6bf34cba1e632c218badcbadd5e3ff3d34ec890fb6" Jul 02 23:13:05.410910 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:05.410883 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:13:05.411300 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:05.410986 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xq5vp" event={"ID":"b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9","Type":"ContainerStarted","Data":"64fb942411faa25032048f278275982b2b1a7a89fc0636122ac3f1bbfcbb4c7c"} Jul 02 23:13:13.265839 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.265804 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-666fdb9888-h4qv4"] Jul 02 23:13:13.266204 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266069 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" containerName="console" Jul 02 23:13:13.266204 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266080 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" containerName="console" Jul 02 23:13:13.266204 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266094 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93c5d449-8ad5-45ef-b99b-afa26ff912d6" containerName="registry" Jul 02 23:13:13.266204 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266099 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="93c5d449-8ad5-45ef-b99b-afa26ff912d6" containerName="registry" Jul 02 23:13:13.266204 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266123 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d84b754-9cac-43f5-8b7b-f75120c70cf6" containerName="console" Jul 02 23:13:13.266204 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266129 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d84b754-9cac-43f5-8b7b-f75120c70cf6" containerName="console" Jul 02 23:13:13.266204 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266144 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" containerName="console" Jul 02 23:13:13.266204 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266150 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" containerName="console" Jul 02 23:13:13.266447 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266211 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="93c5d449-8ad5-45ef-b99b-afa26ff912d6" containerName="registry" Jul 02 23:13:13.266447 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266218 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="713b82f3-e16a-4d3b-b31c-dc74cbd29fa1" containerName="console" Jul 02 23:13:13.266447 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266224 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d84b754-9cac-43f5-8b7b-f75120c70cf6" containerName="console" Jul 02 23:13:13.266447 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.266230 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc7ca2b1-5f0a-4016-8661-40f6c87f54ff" containerName="console" Jul 02 23:13:13.268802 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.268779 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.279765 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.279742 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-666fdb9888-h4qv4"] Jul 02 23:13:13.359833 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.359793 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-config\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.360052 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.359848 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-serving-cert\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.360052 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.359870 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-service-ca\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.360052 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.359930 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-trusted-ca-bundle\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.360052 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.359983 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-oauth-serving-cert\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.360052 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.360046 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-oauth-config\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.360300 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.360091 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htpw5\" (UniqueName: \"kubernetes.io/projected/c8ac7e55-f8e3-4f64-a993-3482cea0472d-kube-api-access-htpw5\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.461195 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.461161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-serving-cert\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.461382 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.461224 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-service-ca\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.461382 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.461262 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-trusted-ca-bundle\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.461382 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.461291 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-oauth-serving-cert\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.461531 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.461445 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-oauth-config\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.461531 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.461514 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htpw5\" (UniqueName: \"kubernetes.io/projected/c8ac7e55-f8e3-4f64-a993-3482cea0472d-kube-api-access-htpw5\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.461624 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.461557 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-config\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.462086 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.462059 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-oauth-serving-cert\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.462250 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.462166 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-config\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.462250 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.462172 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-trusted-ca-bundle\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.462250 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.462167 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-service-ca\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.463805 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.463780 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-serving-cert\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.463967 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.463944 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-oauth-config\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.469681 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.469657 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htpw5\" (UniqueName: \"kubernetes.io/projected/c8ac7e55-f8e3-4f64-a993-3482cea0472d-kube-api-access-htpw5\") pod \"console-666fdb9888-h4qv4\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.577745 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.577717 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:13.703724 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:13.703693 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-666fdb9888-h4qv4"] Jul 02 23:13:13.706626 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:13:13.706604 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ac7e55_f8e3_4f64_a993_3482cea0472d.slice/crio-7a4e6836cffa285c6ee278312aa261dc44741c932d911e256f40e312a4d1bccf WatchSource:0}: Error finding container 7a4e6836cffa285c6ee278312aa261dc44741c932d911e256f40e312a4d1bccf: Status 404 returned error can't find the container with id 7a4e6836cffa285c6ee278312aa261dc44741c932d911e256f40e312a4d1bccf Jul 02 23:13:14.449302 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:14.449257 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666fdb9888-h4qv4" event={"ID":"c8ac7e55-f8e3-4f64-a993-3482cea0472d","Type":"ContainerStarted","Data":"fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855"} Jul 02 23:13:14.449686 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:14.449318 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666fdb9888-h4qv4" event={"ID":"c8ac7e55-f8e3-4f64-a993-3482cea0472d","Type":"ContainerStarted","Data":"7a4e6836cffa285c6ee278312aa261dc44741c932d911e256f40e312a4d1bccf"} Jul 02 23:13:14.468770 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:14.468721 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-666fdb9888-h4qv4" podStartSLOduration=1.468704403 podStartE2EDuration="1.468704403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:13:14.467142016 +0000 UTC m=+186.462390308" watchObservedRunningTime="2026-07-02 23:13:14.468704403 +0000 UTC m=+186.463952693" Jul 02 23:13:23.578742 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:23.578696 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:23.579262 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:23.578801 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:23.583777 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:23.583754 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:24.489235 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:24.489206 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:13:24.533322 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:24.533288 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6585bb9d54-44plb"] Jul 02 23:13:49.561508 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.561433 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6585bb9d54-44plb" podUID="93e22820-d5f6-42dd-893a-7bdd755d5bc9" containerName="console" containerID="cri-o://68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646" gracePeriod=15 Jul 02 23:13:49.804316 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.804294 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:13:49.852252 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852217 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zp6jt\" (UniqueName: \"kubernetes.io/projected/93e22820-d5f6-42dd-893a-7bdd755d5bc9-kube-api-access-zp6jt\") pod \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " Jul 02 23:13:49.852471 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852272 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-serving-cert\") pod \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " Jul 02 23:13:49.852471 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852318 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-trusted-ca-bundle\") pod \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " Jul 02 23:13:49.852471 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852355 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-service-ca\") pod \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " Jul 02 23:13:49.852632 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852471 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-config\") pod \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " Jul 02 23:13:49.852632 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852528 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-oauth-config\") pod \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " Jul 02 23:13:49.852632 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852558 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-oauth-serving-cert\") pod \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\" (UID: \"93e22820-d5f6-42dd-893a-7bdd755d5bc9\") " Jul 02 23:13:49.852786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852737 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-service-ca" (OuterVolumeSpecName: "service-ca") pod "93e22820-d5f6-42dd-893a-7bdd755d5bc9" (UID: "93e22820-d5f6-42dd-893a-7bdd755d5bc9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:13:49.852786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852758 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "93e22820-d5f6-42dd-893a-7bdd755d5bc9" (UID: "93e22820-d5f6-42dd-893a-7bdd755d5bc9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:13:49.852891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852836 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-trusted-ca-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:13:49.852891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852832 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-config" (OuterVolumeSpecName: "console-config") pod "93e22820-d5f6-42dd-893a-7bdd755d5bc9" (UID: "93e22820-d5f6-42dd-893a-7bdd755d5bc9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:13:49.852891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.852853 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:13:49.853087 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.853062 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93e22820-d5f6-42dd-893a-7bdd755d5bc9" (UID: "93e22820-d5f6-42dd-893a-7bdd755d5bc9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:13:49.854521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.854490 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93e22820-d5f6-42dd-893a-7bdd755d5bc9-kube-api-access-zp6jt" (OuterVolumeSpecName: "kube-api-access-zp6jt") pod "93e22820-d5f6-42dd-893a-7bdd755d5bc9" (UID: "93e22820-d5f6-42dd-893a-7bdd755d5bc9"). InnerVolumeSpecName "kube-api-access-zp6jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:13:49.854521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.854508 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93e22820-d5f6-42dd-893a-7bdd755d5bc9" (UID: "93e22820-d5f6-42dd-893a-7bdd755d5bc9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:13:49.854665 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.854522 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93e22820-d5f6-42dd-893a-7bdd755d5bc9" (UID: "93e22820-d5f6-42dd-893a-7bdd755d5bc9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:13:49.953965 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.953926 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:13:49.953965 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.953959 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-oauth-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:13:49.953965 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.953970 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93e22820-d5f6-42dd-893a-7bdd755d5bc9-oauth-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:13:49.954222 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.953978 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp6jt\" (UniqueName: \"kubernetes.io/projected/93e22820-d5f6-42dd-893a-7bdd755d5bc9-kube-api-access-zp6jt\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:13:49.954222 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:49.953988 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93e22820-d5f6-42dd-893a-7bdd755d5bc9-console-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:13:50.584989 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.584954 2570 generic.go:356] "Generic (PLEG): container finished" podID="93e22820-d5f6-42dd-893a-7bdd755d5bc9" containerID="68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646" exitCode=0 Jul 02 23:13:50.585458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.584998 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6585bb9d54-44plb" Jul 02 23:13:50.585458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.585042 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6585bb9d54-44plb" event={"ID":"93e22820-d5f6-42dd-893a-7bdd755d5bc9","Type":"ContainerDied","Data":"68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646"} Jul 02 23:13:50.585458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.585137 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6585bb9d54-44plb" event={"ID":"93e22820-d5f6-42dd-893a-7bdd755d5bc9","Type":"ContainerDied","Data":"e3ae5ba5e918d8e1c97f17e9704fbd78f173e9bae0854d0732fcc36fdf61486f"} Jul 02 23:13:50.585458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.585166 2570 scope.go:117] "RemoveContainer" containerID="68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646" Jul 02 23:13:50.594393 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.594377 2570 scope.go:117] "RemoveContainer" containerID="68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646" Jul 02 23:13:50.594664 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:13:50.594646 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646\": container with ID starting with 68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646 not found: ID does not exist" containerID="68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646" Jul 02 23:13:50.594719 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.594672 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646"} err="failed to get container status \"68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646\": rpc error: code = NotFound desc = could not find container \"68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646\": container with ID starting with 68b1b48933a05cd9608541206e8675e2f47a2ccc72c59edb15641d5eab120646 not found: ID does not exist" Jul 02 23:13:50.608623 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.608595 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6585bb9d54-44plb"] Jul 02 23:13:50.612685 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:50.612661 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6585bb9d54-44plb"] Jul 02 23:13:52.591510 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:13:52.591476 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93e22820-d5f6-42dd-893a-7bdd755d5bc9" path="/var/lib/kubelet/pods/93e22820-d5f6-42dd-893a-7bdd755d5bc9/volumes" Jul 02 23:14:02.937801 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.937766 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg"] Jul 02 23:14:02.938301 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.938054 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93e22820-d5f6-42dd-893a-7bdd755d5bc9" containerName="console" Jul 02 23:14:02.938301 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.938070 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="93e22820-d5f6-42dd-893a-7bdd755d5bc9" containerName="console" Jul 02 23:14:02.938301 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.938131 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="93e22820-d5f6-42dd-893a-7bdd755d5bc9" containerName="console" Jul 02 23:14:02.940045 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.940021 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:02.943127 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.943105 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:14:02.943229 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.943105 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:14:02.944385 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.944364 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4q69t\"" Jul 02 23:14:02.953104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:02.953077 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg"] Jul 02 23:14:03.052514 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.052471 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn7b8\" (UniqueName: \"kubernetes.io/projected/768786d2-4fc7-4418-9eaf-4df239203cca-kube-api-access-gn7b8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.052514 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.052521 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.052718 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.052578 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.153796 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.153752 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.153917 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.153818 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn7b8\" (UniqueName: \"kubernetes.io/projected/768786d2-4fc7-4418-9eaf-4df239203cca-kube-api-access-gn7b8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.153917 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.153851 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.154173 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.154156 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.154274 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.154256 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.163412 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.163383 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn7b8\" (UniqueName: \"kubernetes.io/projected/768786d2-4fc7-4418-9eaf-4df239203cca-kube-api-access-gn7b8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.249462 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.249355 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:03.384714 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.384680 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg"] Jul 02 23:14:03.388087 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:14:03.388061 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768786d2_4fc7_4418_9eaf_4df239203cca.slice/crio-442a5438fc3526231797114c95e4f28d271fb5782194f5082bb334231334d9dd WatchSource:0}: Error finding container 442a5438fc3526231797114c95e4f28d271fb5782194f5082bb334231334d9dd: Status 404 returned error can't find the container with id 442a5438fc3526231797114c95e4f28d271fb5782194f5082bb334231334d9dd Jul 02 23:14:03.634510 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:03.634471 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" event={"ID":"768786d2-4fc7-4418-9eaf-4df239203cca","Type":"ContainerStarted","Data":"442a5438fc3526231797114c95e4f28d271fb5782194f5082bb334231334d9dd"} Jul 02 23:14:10.663307 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:10.663274 2570 generic.go:356] "Generic (PLEG): container finished" podID="768786d2-4fc7-4418-9eaf-4df239203cca" containerID="e28d8ca48aef3e72a9a31116617a456d7ec8e8042305652fdf577517e63a40a4" exitCode=0 Jul 02 23:14:10.663774 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:10.663316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" event={"ID":"768786d2-4fc7-4418-9eaf-4df239203cca","Type":"ContainerDied","Data":"e28d8ca48aef3e72a9a31116617a456d7ec8e8042305652fdf577517e63a40a4"} Jul 02 23:14:13.678729 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:13.678688 2570 generic.go:356] "Generic (PLEG): container finished" podID="768786d2-4fc7-4418-9eaf-4df239203cca" containerID="39dd58c605c81f15d74392113d5f397e0af662e830bc52f563c90ec0410b4303" exitCode=0 Jul 02 23:14:13.679225 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:13.678765 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" event={"ID":"768786d2-4fc7-4418-9eaf-4df239203cca","Type":"ContainerDied","Data":"39dd58c605c81f15d74392113d5f397e0af662e830bc52f563c90ec0410b4303"} Jul 02 23:14:20.709379 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:20.709344 2570 generic.go:356] "Generic (PLEG): container finished" podID="768786d2-4fc7-4418-9eaf-4df239203cca" containerID="705b2438255acf9ac3c4b8518b8fad59836ed3c71bfaab5d6469428520a4f7c2" exitCode=0 Jul 02 23:14:20.709379 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:20.709386 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" event={"ID":"768786d2-4fc7-4418-9eaf-4df239203cca","Type":"ContainerDied","Data":"705b2438255acf9ac3c4b8518b8fad59836ed3c71bfaab5d6469428520a4f7c2"} Jul 02 23:14:21.834203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:21.834165 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:22.005325 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.005234 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-util\") pod \"768786d2-4fc7-4418-9eaf-4df239203cca\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " Jul 02 23:14:22.005325 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.005275 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-bundle\") pod \"768786d2-4fc7-4418-9eaf-4df239203cca\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " Jul 02 23:14:22.005325 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.005309 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn7b8\" (UniqueName: \"kubernetes.io/projected/768786d2-4fc7-4418-9eaf-4df239203cca-kube-api-access-gn7b8\") pod \"768786d2-4fc7-4418-9eaf-4df239203cca\" (UID: \"768786d2-4fc7-4418-9eaf-4df239203cca\") " Jul 02 23:14:22.006067 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.006038 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-bundle" (OuterVolumeSpecName: "bundle") pod "768786d2-4fc7-4418-9eaf-4df239203cca" (UID: "768786d2-4fc7-4418-9eaf-4df239203cca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:22.007490 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.007468 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768786d2-4fc7-4418-9eaf-4df239203cca-kube-api-access-gn7b8" (OuterVolumeSpecName: "kube-api-access-gn7b8") pod "768786d2-4fc7-4418-9eaf-4df239203cca" (UID: "768786d2-4fc7-4418-9eaf-4df239203cca"). InnerVolumeSpecName "kube-api-access-gn7b8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:22.009813 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.009785 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-util" (OuterVolumeSpecName: "util") pod "768786d2-4fc7-4418-9eaf-4df239203cca" (UID: "768786d2-4fc7-4418-9eaf-4df239203cca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:22.106627 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.106573 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-util\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:22.106627 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.106618 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/768786d2-4fc7-4418-9eaf-4df239203cca-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:22.106627 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.106629 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn7b8\" (UniqueName: \"kubernetes.io/projected/768786d2-4fc7-4418-9eaf-4df239203cca-kube-api-access-gn7b8\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:22.718258 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.718227 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" event={"ID":"768786d2-4fc7-4418-9eaf-4df239203cca","Type":"ContainerDied","Data":"442a5438fc3526231797114c95e4f28d271fb5782194f5082bb334231334d9dd"} Jul 02 23:14:22.718421 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.718278 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="442a5438fc3526231797114c95e4f28d271fb5782194f5082bb334231334d9dd" Jul 02 23:14:22.718421 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:22.718238 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jsdjg" Jul 02 23:14:25.962157 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:25.962124 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2"] Jul 02 23:14:25.962656 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:25.962414 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768786d2-4fc7-4418-9eaf-4df239203cca" containerName="extract" Jul 02 23:14:25.962656 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:25.962425 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="768786d2-4fc7-4418-9eaf-4df239203cca" containerName="extract" Jul 02 23:14:25.962656 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:25.962441 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768786d2-4fc7-4418-9eaf-4df239203cca" containerName="pull" Jul 02 23:14:25.962656 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:25.962447 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="768786d2-4fc7-4418-9eaf-4df239203cca" containerName="pull" Jul 02 23:14:25.962656 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:25.962454 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768786d2-4fc7-4418-9eaf-4df239203cca" containerName="util" Jul 02 23:14:25.962656 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:25.962459 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="768786d2-4fc7-4418-9eaf-4df239203cca" containerName="util" Jul 02 23:14:25.962656 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:25.962499 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="768786d2-4fc7-4418-9eaf-4df239203cca" containerName="extract" Jul 02 23:14:26.020795 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.020763 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2"] Jul 02 23:14:26.020949 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.020859 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" Jul 02 23:14:26.025031 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.025006 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 02 23:14:26.025031 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.025026 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:14:26.025228 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.025043 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-bcqbx\"" Jul 02 23:14:26.142643 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.142599 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4830e06e-2fc2-441f-a9d1-2e08c0b9357f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzrb2\" (UID: \"4830e06e-2fc2-441f-a9d1-2e08c0b9357f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" Jul 02 23:14:26.142821 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.142661 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cfpl\" (UniqueName: \"kubernetes.io/projected/4830e06e-2fc2-441f-a9d1-2e08c0b9357f-kube-api-access-9cfpl\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzrb2\" (UID: \"4830e06e-2fc2-441f-a9d1-2e08c0b9357f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" Jul 02 23:14:26.243487 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.243404 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4830e06e-2fc2-441f-a9d1-2e08c0b9357f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzrb2\" (UID: \"4830e06e-2fc2-441f-a9d1-2e08c0b9357f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" Jul 02 23:14:26.243487 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.243455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cfpl\" (UniqueName: \"kubernetes.io/projected/4830e06e-2fc2-441f-a9d1-2e08c0b9357f-kube-api-access-9cfpl\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzrb2\" (UID: \"4830e06e-2fc2-441f-a9d1-2e08c0b9357f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" Jul 02 23:14:26.243787 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.243766 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4830e06e-2fc2-441f-a9d1-2e08c0b9357f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzrb2\" (UID: \"4830e06e-2fc2-441f-a9d1-2e08c0b9357f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" Jul 02 23:14:26.256313 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.256293 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cfpl\" (UniqueName: \"kubernetes.io/projected/4830e06e-2fc2-441f-a9d1-2e08c0b9357f-kube-api-access-9cfpl\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzrb2\" (UID: \"4830e06e-2fc2-441f-a9d1-2e08c0b9357f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" Jul 02 23:14:26.331379 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.331315 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" Jul 02 23:14:26.484931 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.484891 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2"] Jul 02 23:14:26.489961 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:14:26.489934 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4830e06e_2fc2_441f_a9d1_2e08c0b9357f.slice/crio-bd36b2c8b51677c315c75fb6486824b10a106566af651436d16656e94e6749dc WatchSource:0}: Error finding container bd36b2c8b51677c315c75fb6486824b10a106566af651436d16656e94e6749dc: Status 404 returned error can't find the container with id bd36b2c8b51677c315c75fb6486824b10a106566af651436d16656e94e6749dc Jul 02 23:14:26.737573 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:26.737543 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" event={"ID":"4830e06e-2fc2-441f-a9d1-2e08c0b9357f","Type":"ContainerStarted","Data":"bd36b2c8b51677c315c75fb6486824b10a106566af651436d16656e94e6749dc"} Jul 02 23:14:28.747489 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:28.747444 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" event={"ID":"4830e06e-2fc2-441f-a9d1-2e08c0b9357f","Type":"ContainerStarted","Data":"e29b9302643573962f12ab266c4884d2ce2589cc41ed5b5b3e65cede0caa7524"} Jul 02 23:14:29.771721 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:29.771660 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzrb2" podStartSLOduration=2.647215604 podStartE2EDuration="4.771643385s" totalImagesPullingTime="2.124427781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:29.769731584 +0000 UTC m=+261.764979876" watchObservedRunningTime="2026-07-02 23:14:29.771643385 +0000 UTC m=+261.766891676" Jul 02 23:14:30.440336 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.440301 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj"] Jul 02 23:14:30.442606 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.442588 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.445982 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.445957 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:14:30.446441 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.446420 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:14:30.447300 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.447287 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4q69t\"" Jul 02 23:14:30.456846 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.456823 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj"] Jul 02 23:14:30.578089 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.578055 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmz6t\" (UniqueName: \"kubernetes.io/projected/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-kube-api-access-bmz6t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.578343 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.578103 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.578343 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.578224 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.678938 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.678897 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.678938 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.678939 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmz6t\" (UniqueName: \"kubernetes.io/projected/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-kube-api-access-bmz6t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.679131 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.678987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.679284 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.679265 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.679325 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.679299 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.688331 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.688309 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmz6t\" (UniqueName: \"kubernetes.io/projected/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-kube-api-access-bmz6t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.751989 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.751909 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:30.880848 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:30.880744 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj"] Jul 02 23:14:30.883390 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:14:30.883361 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e96fe1_6152_4f9f_904f_9b3f3ca85136.slice/crio-d8285347899cfdd8914df139398c7bd7a2b5f27e00a4b7880e4ebe0c455aeab0 WatchSource:0}: Error finding container d8285347899cfdd8914df139398c7bd7a2b5f27e00a4b7880e4ebe0c455aeab0: Status 404 returned error can't find the container with id d8285347899cfdd8914df139398c7bd7a2b5f27e00a4b7880e4ebe0c455aeab0 Jul 02 23:14:31.760770 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:31.760736 2570 generic.go:356] "Generic (PLEG): container finished" podID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerID="591db3f12eedf9a961cc5beb839fb8585f14a38073f3cee388513aa3010b0c73" exitCode=0 Jul 02 23:14:31.760934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:31.760800 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" event={"ID":"e2e96fe1-6152-4f9f-904f-9b3f3ca85136","Type":"ContainerDied","Data":"591db3f12eedf9a961cc5beb839fb8585f14a38073f3cee388513aa3010b0c73"} Jul 02 23:14:31.760934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:31.760834 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" event={"ID":"e2e96fe1-6152-4f9f-904f-9b3f3ca85136","Type":"ContainerStarted","Data":"d8285347899cfdd8914df139398c7bd7a2b5f27e00a4b7880e4ebe0c455aeab0"} Jul 02 23:14:32.175253 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.175216 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zmk86"] Jul 02 23:14:32.177416 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.177398 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" Jul 02 23:14:32.180246 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.180218 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 23:14:32.180562 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.180544 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qz8wn\"" Jul 02 23:14:32.180661 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.180580 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 23:14:32.188417 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.188397 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zmk86"] Jul 02 23:14:32.292069 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.292037 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5715c6d3-50d6-4a79-9503-08ba6dc547a4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zmk86\" (UID: \"5715c6d3-50d6-4a79-9503-08ba6dc547a4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" Jul 02 23:14:32.292260 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.292090 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r5f2\" (UniqueName: \"kubernetes.io/projected/5715c6d3-50d6-4a79-9503-08ba6dc547a4-kube-api-access-9r5f2\") pod \"cert-manager-cainjector-559cdd8c87-zmk86\" (UID: \"5715c6d3-50d6-4a79-9503-08ba6dc547a4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" Jul 02 23:14:32.392788 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.392747 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r5f2\" (UniqueName: \"kubernetes.io/projected/5715c6d3-50d6-4a79-9503-08ba6dc547a4-kube-api-access-9r5f2\") pod \"cert-manager-cainjector-559cdd8c87-zmk86\" (UID: \"5715c6d3-50d6-4a79-9503-08ba6dc547a4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" Jul 02 23:14:32.392981 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.392846 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5715c6d3-50d6-4a79-9503-08ba6dc547a4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zmk86\" (UID: \"5715c6d3-50d6-4a79-9503-08ba6dc547a4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" Jul 02 23:14:32.401608 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.401573 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5715c6d3-50d6-4a79-9503-08ba6dc547a4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zmk86\" (UID: \"5715c6d3-50d6-4a79-9503-08ba6dc547a4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" Jul 02 23:14:32.401886 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.401857 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r5f2\" (UniqueName: \"kubernetes.io/projected/5715c6d3-50d6-4a79-9503-08ba6dc547a4-kube-api-access-9r5f2\") pod \"cert-manager-cainjector-559cdd8c87-zmk86\" (UID: \"5715c6d3-50d6-4a79-9503-08ba6dc547a4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" Jul 02 23:14:32.500926 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.500853 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" Jul 02 23:14:32.646342 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.646285 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zmk86"] Jul 02 23:14:32.648544 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:14:32.648515 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5715c6d3_50d6_4a79_9503_08ba6dc547a4.slice/crio-221203641ff67b8bc950a7b980a2ca72c87932c9a001d1cf2aaddc9c4f903ca0 WatchSource:0}: Error finding container 221203641ff67b8bc950a7b980a2ca72c87932c9a001d1cf2aaddc9c4f903ca0: Status 404 returned error can't find the container with id 221203641ff67b8bc950a7b980a2ca72c87932c9a001d1cf2aaddc9c4f903ca0 Jul 02 23:14:32.766506 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:32.766422 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" event={"ID":"5715c6d3-50d6-4a79-9503-08ba6dc547a4","Type":"ContainerStarted","Data":"221203641ff67b8bc950a7b980a2ca72c87932c9a001d1cf2aaddc9c4f903ca0"} Jul 02 23:14:33.773662 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:33.773624 2570 generic.go:356] "Generic (PLEG): container finished" podID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerID="c8b949cdf36d92e73f00cac7e5704cfa087eddeb4f05e9b97d9272bbab6c7f1f" exitCode=0 Jul 02 23:14:33.773662 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:33.773670 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" event={"ID":"e2e96fe1-6152-4f9f-904f-9b3f3ca85136","Type":"ContainerDied","Data":"c8b949cdf36d92e73f00cac7e5704cfa087eddeb4f05e9b97d9272bbab6c7f1f"} Jul 02 23:14:34.781262 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:34.781226 2570 generic.go:356] "Generic (PLEG): container finished" podID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerID="64f3a65e845269f2912434c3b80f36e1ccfea3e17911c0d4ea55ebee10b132d6" exitCode=0 Jul 02 23:14:34.781746 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:34.781338 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" event={"ID":"e2e96fe1-6152-4f9f-904f-9b3f3ca85136","Type":"ContainerDied","Data":"64f3a65e845269f2912434c3b80f36e1ccfea3e17911c0d4ea55ebee10b132d6"} Jul 02 23:14:35.787298 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:35.787265 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" event={"ID":"5715c6d3-50d6-4a79-9503-08ba6dc547a4","Type":"ContainerStarted","Data":"0618a28aca766e8c060a0709607a305c54e52406825e84cada7de709d7db3184"} Jul 02 23:14:35.806263 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:35.806198 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zmk86" podStartSLOduration=0.879490575 podStartE2EDuration="3.806164899s" totalImagesPullingTime="2.926674324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:35.804661484 +0000 UTC m=+267.799909776" watchObservedRunningTime="2026-07-02 23:14:35.806164899 +0000 UTC m=+267.801413190" Jul 02 23:14:35.937277 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:35.937253 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:36.027609 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.027579 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmz6t\" (UniqueName: \"kubernetes.io/projected/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-kube-api-access-bmz6t\") pod \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " Jul 02 23:14:36.027790 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.027627 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-bundle\") pod \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " Jul 02 23:14:36.027790 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.027656 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-util\") pod \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\" (UID: \"e2e96fe1-6152-4f9f-904f-9b3f3ca85136\") " Jul 02 23:14:36.028119 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.028091 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-bundle" (OuterVolumeSpecName: "bundle") pod "e2e96fe1-6152-4f9f-904f-9b3f3ca85136" (UID: "e2e96fe1-6152-4f9f-904f-9b3f3ca85136"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:36.029700 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.029678 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-kube-api-access-bmz6t" (OuterVolumeSpecName: "kube-api-access-bmz6t") pod "e2e96fe1-6152-4f9f-904f-9b3f3ca85136" (UID: "e2e96fe1-6152-4f9f-904f-9b3f3ca85136"). InnerVolumeSpecName "kube-api-access-bmz6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:36.033319 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.033292 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-util" (OuterVolumeSpecName: "util") pod "e2e96fe1-6152-4f9f-904f-9b3f3ca85136" (UID: "e2e96fe1-6152-4f9f-904f-9b3f3ca85136"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:36.128469 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.128384 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-util\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:36.128469 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.128415 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmz6t\" (UniqueName: \"kubernetes.io/projected/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-kube-api-access-bmz6t\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:36.128469 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.128427 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2e96fe1-6152-4f9f-904f-9b3f3ca85136-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:36.148425 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.148393 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tz8d7"] Jul 02 23:14:36.148675 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.148663 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerName="pull" Jul 02 23:14:36.148716 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.148677 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerName="pull" Jul 02 23:14:36.148716 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.148684 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerName="extract" Jul 02 23:14:36.148716 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.148689 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerName="extract" Jul 02 23:14:36.148716 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.148707 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerName="util" Jul 02 23:14:36.148716 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.148712 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerName="util" Jul 02 23:14:36.148862 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.148756 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2e96fe1-6152-4f9f-904f-9b3f3ca85136" containerName="extract" Jul 02 23:14:36.150611 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.150595 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.153569 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.153545 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-j2r42\"" Jul 02 23:14:36.161634 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.161612 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tz8d7"] Jul 02 23:14:36.229440 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.229408 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d6g6\" (UniqueName: \"kubernetes.io/projected/a69df88c-d4b2-45cc-8b6a-0c3ddd548173-kube-api-access-6d6g6\") pod \"cert-manager-webhook-6486ffd997-tz8d7\" (UID: \"a69df88c-d4b2-45cc-8b6a-0c3ddd548173\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.229596 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.229450 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a69df88c-d4b2-45cc-8b6a-0c3ddd548173-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tz8d7\" (UID: \"a69df88c-d4b2-45cc-8b6a-0c3ddd548173\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.329960 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.329925 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d6g6\" (UniqueName: \"kubernetes.io/projected/a69df88c-d4b2-45cc-8b6a-0c3ddd548173-kube-api-access-6d6g6\") pod \"cert-manager-webhook-6486ffd997-tz8d7\" (UID: \"a69df88c-d4b2-45cc-8b6a-0c3ddd548173\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.330125 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.329968 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a69df88c-d4b2-45cc-8b6a-0c3ddd548173-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tz8d7\" (UID: \"a69df88c-d4b2-45cc-8b6a-0c3ddd548173\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.338756 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.338723 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d6g6\" (UniqueName: \"kubernetes.io/projected/a69df88c-d4b2-45cc-8b6a-0c3ddd548173-kube-api-access-6d6g6\") pod \"cert-manager-webhook-6486ffd997-tz8d7\" (UID: \"a69df88c-d4b2-45cc-8b6a-0c3ddd548173\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.338872 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.338753 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a69df88c-d4b2-45cc-8b6a-0c3ddd548173-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tz8d7\" (UID: \"a69df88c-d4b2-45cc-8b6a-0c3ddd548173\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.459903 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.459814 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.584099 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.584065 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tz8d7"] Jul 02 23:14:36.586425 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:14:36.586394 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda69df88c_d4b2_45cc_8b6a_0c3ddd548173.slice/crio-f4e6809ca732ccef0710cbd47b95abefae4559e92e2d98c772dba2e11e0231bb WatchSource:0}: Error finding container f4e6809ca732ccef0710cbd47b95abefae4559e92e2d98c772dba2e11e0231bb: Status 404 returned error can't find the container with id f4e6809ca732ccef0710cbd47b95abefae4559e92e2d98c772dba2e11e0231bb Jul 02 23:14:36.793303 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.793208 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" event={"ID":"a69df88c-d4b2-45cc-8b6a-0c3ddd548173","Type":"ContainerStarted","Data":"146b0533a32a754b836eb89073cff1b0494d47101d1734463fe9b937070cd93c"} Jul 02 23:14:36.793303 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.793250 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" event={"ID":"a69df88c-d4b2-45cc-8b6a-0c3ddd548173","Type":"ContainerStarted","Data":"f4e6809ca732ccef0710cbd47b95abefae4559e92e2d98c772dba2e11e0231bb"} Jul 02 23:14:36.793303 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.793299 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:36.795141 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.795115 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" Jul 02 23:14:36.795289 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.795154 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw7ffj" event={"ID":"e2e96fe1-6152-4f9f-904f-9b3f3ca85136","Type":"ContainerDied","Data":"d8285347899cfdd8914df139398c7bd7a2b5f27e00a4b7880e4ebe0c455aeab0"} Jul 02 23:14:36.795289 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.795215 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d8285347899cfdd8914df139398c7bd7a2b5f27e00a4b7880e4ebe0c455aeab0" Jul 02 23:14:36.812043 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:36.811990 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" podStartSLOduration=0.811972987 podStartE2EDuration="811.972987ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:36.810788744 +0000 UTC m=+268.806037038" watchObservedRunningTime="2026-07-02 23:14:36.811972987 +0000 UTC m=+268.807221277" Jul 02 23:14:42.801888 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:42.801857 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tz8d7" Jul 02 23:14:50.278911 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.278873 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5ff69"] Jul 02 23:14:50.303320 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.303286 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5ff69"] Jul 02 23:14:50.303488 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.303408 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5ff69" Jul 02 23:14:50.306438 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.306419 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9jb58\"" Jul 02 23:14:50.344720 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.344678 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjn2w\" (UniqueName: \"kubernetes.io/projected/a18cc7ac-be0c-4545-8f5e-13515e6b6281-kube-api-access-tjn2w\") pod \"cert-manager-55b4d954d5-5ff69\" (UID: \"a18cc7ac-be0c-4545-8f5e-13515e6b6281\") " pod="cert-manager/cert-manager-55b4d954d5-5ff69" Jul 02 23:14:50.344897 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.344785 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a18cc7ac-be0c-4545-8f5e-13515e6b6281-bound-sa-token\") pod \"cert-manager-55b4d954d5-5ff69\" (UID: \"a18cc7ac-be0c-4545-8f5e-13515e6b6281\") " pod="cert-manager/cert-manager-55b4d954d5-5ff69" Jul 02 23:14:50.445395 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.445357 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a18cc7ac-be0c-4545-8f5e-13515e6b6281-bound-sa-token\") pod \"cert-manager-55b4d954d5-5ff69\" (UID: \"a18cc7ac-be0c-4545-8f5e-13515e6b6281\") " pod="cert-manager/cert-manager-55b4d954d5-5ff69" Jul 02 23:14:50.445570 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.445515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjn2w\" (UniqueName: \"kubernetes.io/projected/a18cc7ac-be0c-4545-8f5e-13515e6b6281-kube-api-access-tjn2w\") pod \"cert-manager-55b4d954d5-5ff69\" (UID: \"a18cc7ac-be0c-4545-8f5e-13515e6b6281\") " pod="cert-manager/cert-manager-55b4d954d5-5ff69" Jul 02 23:14:50.454394 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.454368 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a18cc7ac-be0c-4545-8f5e-13515e6b6281-bound-sa-token\") pod \"cert-manager-55b4d954d5-5ff69\" (UID: \"a18cc7ac-be0c-4545-8f5e-13515e6b6281\") " pod="cert-manager/cert-manager-55b4d954d5-5ff69" Jul 02 23:14:50.454967 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.454947 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjn2w\" (UniqueName: \"kubernetes.io/projected/a18cc7ac-be0c-4545-8f5e-13515e6b6281-kube-api-access-tjn2w\") pod \"cert-manager-55b4d954d5-5ff69\" (UID: \"a18cc7ac-be0c-4545-8f5e-13515e6b6281\") " pod="cert-manager/cert-manager-55b4d954d5-5ff69" Jul 02 23:14:50.612491 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.612451 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5ff69" Jul 02 23:14:50.763689 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.763665 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5ff69"] Jul 02 23:14:50.765790 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:14:50.765761 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda18cc7ac_be0c_4545_8f5e_13515e6b6281.slice/crio-be4d582acc15c5634197b5c63fc1c34aea62d179d38b4dd95e7ca6bf29d71969 WatchSource:0}: Error finding container be4d582acc15c5634197b5c63fc1c34aea62d179d38b4dd95e7ca6bf29d71969: Status 404 returned error can't find the container with id be4d582acc15c5634197b5c63fc1c34aea62d179d38b4dd95e7ca6bf29d71969 Jul 02 23:14:50.860829 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:50.860797 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5ff69" event={"ID":"a18cc7ac-be0c-4545-8f5e-13515e6b6281","Type":"ContainerStarted","Data":"be4d582acc15c5634197b5c63fc1c34aea62d179d38b4dd95e7ca6bf29d71969"} Jul 02 23:14:51.866931 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:51.866892 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5ff69" event={"ID":"a18cc7ac-be0c-4545-8f5e-13515e6b6281","Type":"ContainerStarted","Data":"abeefb879edd051acc88c5c5179aeefda952a2eddad7c4325e024f42117f5f0b"} Jul 02 23:14:51.901903 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:51.901848 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5ff69" podStartSLOduration=1.901831493 podStartE2EDuration="1.901831493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:14:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:14:51.901299709 +0000 UTC m=+283.896548000" watchObservedRunningTime="2026-07-02 23:14:51.901831493 +0000 UTC m=+283.897079784" Jul 02 23:14:53.886115 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.886079 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd"] Jul 02 23:14:53.888814 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.888797 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:53.893246 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.892483 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 23:14:53.894168 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.893648 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 23:14:53.894168 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.893925 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4q69t\"" Jul 02 23:14:53.899704 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.899680 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd"] Jul 02 23:14:53.971552 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.971513 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82qst\" (UniqueName: \"kubernetes.io/projected/77eb96d0-97cc-47d5-80b9-67af7fc494a3-kube-api-access-82qst\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:53.971552 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.971558 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:53.971784 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:53.971584 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:54.072743 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.072696 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82qst\" (UniqueName: \"kubernetes.io/projected/77eb96d0-97cc-47d5-80b9-67af7fc494a3-kube-api-access-82qst\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:54.072743 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.072744 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:54.073004 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.072786 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:54.073264 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.073241 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:54.073342 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.073260 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:54.083822 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.083785 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82qst\" (UniqueName: \"kubernetes.io/projected/77eb96d0-97cc-47d5-80b9-67af7fc494a3-kube-api-access-82qst\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:54.203133 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.203043 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:54.339309 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.339214 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd"] Jul 02 23:14:54.341637 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:14:54.341608 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77eb96d0_97cc_47d5_80b9_67af7fc494a3.slice/crio-c68cc82d088597a7a596eb43ab949b57ccaf8bc449c128c9177c1bee705283a4 WatchSource:0}: Error finding container c68cc82d088597a7a596eb43ab949b57ccaf8bc449c128c9177c1bee705283a4: Status 404 returned error can't find the container with id c68cc82d088597a7a596eb43ab949b57ccaf8bc449c128c9177c1bee705283a4 Jul 02 23:14:54.881086 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.881052 2570 generic.go:356] "Generic (PLEG): container finished" podID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerID="6cec0df63ce4e7157ce41ebdb579bf5ec159de16cb1f86793dbbc1be6631cbdf" exitCode=0 Jul 02 23:14:54.881289 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.881115 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" event={"ID":"77eb96d0-97cc-47d5-80b9-67af7fc494a3","Type":"ContainerDied","Data":"6cec0df63ce4e7157ce41ebdb579bf5ec159de16cb1f86793dbbc1be6631cbdf"} Jul 02 23:14:54.881289 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:54.881165 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" event={"ID":"77eb96d0-97cc-47d5-80b9-67af7fc494a3","Type":"ContainerStarted","Data":"c68cc82d088597a7a596eb43ab949b57ccaf8bc449c128c9177c1bee705283a4"} Jul 02 23:14:55.887599 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:55.887562 2570 generic.go:356] "Generic (PLEG): container finished" podID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerID="81f44bf59cf83a7ab0bbb4535cebacf6cbbfbd92d6b308e92cc011a5a98ae406" exitCode=0 Jul 02 23:14:55.887982 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:55.887652 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" event={"ID":"77eb96d0-97cc-47d5-80b9-67af7fc494a3","Type":"ContainerDied","Data":"81f44bf59cf83a7ab0bbb4535cebacf6cbbfbd92d6b308e92cc011a5a98ae406"} Jul 02 23:14:56.894823 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:56.894789 2570 generic.go:356] "Generic (PLEG): container finished" podID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerID="c8eb9f946bec2f962394e3a7b328f737b2aacc0d19a423228c50be03399d5f91" exitCode=0 Jul 02 23:14:56.895235 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:56.894874 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" event={"ID":"77eb96d0-97cc-47d5-80b9-67af7fc494a3","Type":"ContainerDied","Data":"c8eb9f946bec2f962394e3a7b328f737b2aacc0d19a423228c50be03399d5f91"} Jul 02 23:14:58.022497 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.022468 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:58.106135 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.106096 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-bundle\") pod \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " Jul 02 23:14:58.106355 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.106174 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82qst\" (UniqueName: \"kubernetes.io/projected/77eb96d0-97cc-47d5-80b9-67af7fc494a3-kube-api-access-82qst\") pod \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " Jul 02 23:14:58.106355 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.106244 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-util\") pod \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\" (UID: \"77eb96d0-97cc-47d5-80b9-67af7fc494a3\") " Jul 02 23:14:58.106884 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.106847 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-bundle" (OuterVolumeSpecName: "bundle") pod "77eb96d0-97cc-47d5-80b9-67af7fc494a3" (UID: "77eb96d0-97cc-47d5-80b9-67af7fc494a3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:58.108305 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.108283 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77eb96d0-97cc-47d5-80b9-67af7fc494a3-kube-api-access-82qst" (OuterVolumeSpecName: "kube-api-access-82qst") pod "77eb96d0-97cc-47d5-80b9-67af7fc494a3" (UID: "77eb96d0-97cc-47d5-80b9-67af7fc494a3"). InnerVolumeSpecName "kube-api-access-82qst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:14:58.112199 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.112166 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-util" (OuterVolumeSpecName: "util") pod "77eb96d0-97cc-47d5-80b9-67af7fc494a3" (UID: "77eb96d0-97cc-47d5-80b9-67af7fc494a3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:14:58.206861 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.206766 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:58.206861 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.206799 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82qst\" (UniqueName: \"kubernetes.io/projected/77eb96d0-97cc-47d5-80b9-67af7fc494a3-kube-api-access-82qst\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:58.206861 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.206814 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77eb96d0-97cc-47d5-80b9-67af7fc494a3-util\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:14:58.905102 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.905022 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" event={"ID":"77eb96d0-97cc-47d5-80b9-67af7fc494a3","Type":"ContainerDied","Data":"c68cc82d088597a7a596eb43ab949b57ccaf8bc449c128c9177c1bee705283a4"} Jul 02 23:14:58.905102 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.905063 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwr9zd" Jul 02 23:14:58.905102 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:14:58.905094 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c68cc82d088597a7a596eb43ab949b57ccaf8bc449c128c9177c1bee705283a4" Jul 02 23:15:05.466495 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.466456 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc"] Jul 02 23:15:05.467006 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.466762 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerName="pull" Jul 02 23:15:05.467006 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.466773 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerName="pull" Jul 02 23:15:05.467006 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.466785 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerName="extract" Jul 02 23:15:05.467006 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.466790 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerName="extract" Jul 02 23:15:05.467006 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.466803 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerName="util" Jul 02 23:15:05.467006 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.466808 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerName="util" Jul 02 23:15:05.467006 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.466856 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="77eb96d0-97cc-47d5-80b9-67af7fc494a3" containerName="extract" Jul 02 23:15:05.470110 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.470090 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.473102 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.473081 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 23:15:05.473294 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.473168 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 23:15:05.473508 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.473475 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 23:15:05.473605 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.473588 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n47wn\"" Jul 02 23:15:05.475090 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.475075 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 23:15:05.490057 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.490029 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc"] Jul 02 23:15:05.566624 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.566586 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.566805 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.566631 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-manifests\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.566805 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.566660 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnkln\" (UniqueName: \"kubernetes.io/projected/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-kube-api-access-cnkln\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.566805 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.566697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-tmp\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.566805 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.566723 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-webhook-cert\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.667121 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.667084 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-manifests\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.667339 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.667128 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnkln\" (UniqueName: \"kubernetes.io/projected/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-kube-api-access-cnkln\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.667339 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.667271 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-tmp\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.667472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.667351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-webhook-cert\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.667472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.667401 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.667602 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.667584 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-manifests\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.669440 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.669417 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-tmp\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.669745 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.669728 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.669802 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.669784 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-webhook-cert\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.680163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.680138 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnkln\" (UniqueName: \"kubernetes.io/projected/7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf-kube-api-access-cnkln\") pod \"opendatahub-operator-controller-manager-79946c878f-zm7zc\" (UID: \"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf\") " pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.780104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.780012 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:05.921236 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.921209 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc"] Jul 02 23:15:05.923421 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:15:05.923388 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc8f9e2_9cfc_4c38_a83b_e68a0f76d6bf.slice/crio-68a5998f4a758e8de04b6b2424fea854f50d88886d7282468df90374aa38e75f WatchSource:0}: Error finding container 68a5998f4a758e8de04b6b2424fea854f50d88886d7282468df90374aa38e75f: Status 404 returned error can't find the container with id 68a5998f4a758e8de04b6b2424fea854f50d88886d7282468df90374aa38e75f Jul 02 23:15:05.937062 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:05.937022 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" event={"ID":"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf","Type":"ContainerStarted","Data":"68a5998f4a758e8de04b6b2424fea854f50d88886d7282468df90374aa38e75f"} Jul 02 23:15:08.999433 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:08.999401 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:15:09.000501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:08.999710 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:15:09.006490 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:09.006473 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:15:09.006873 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:09.006854 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:15:09.013483 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:09.013466 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 23:15:09.957604 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:09.957565 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" event={"ID":"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf","Type":"ContainerStarted","Data":"b7de5da2673b68126980ba206f7810131594cb7fd96f1a3b019bc85f219037ed"} Jul 02 23:15:10.963576 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:10.963528 2570 generic.go:356] "Generic (PLEG): container finished" podID="7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf" containerID="b7de5da2673b68126980ba206f7810131594cb7fd96f1a3b019bc85f219037ed" exitCode=0 Jul 02 23:15:10.964001 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:10.963598 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" event={"ID":"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf","Type":"ContainerDied","Data":"b7de5da2673b68126980ba206f7810131594cb7fd96f1a3b019bc85f219037ed"} Jul 02 23:15:10.964136 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:10.964118 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:15:11.970638 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:11.970608 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" event={"ID":"7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf","Type":"ContainerStarted","Data":"761244db08444233aae308885ea1297f7d174fbdc71ccc0f45308ca814454499"} Jul 02 23:15:11.971034 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:11.970722 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:12.000840 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:11.996329 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" podStartSLOduration=2.508764846 podStartE2EDuration="6.996310333s" totalImagesPullingTime="3.449214653s" totalInitContainerRuntime="1.038330834s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:11.993613574 +0000 UTC m=+303.988861865" watchObservedRunningTime="2026-07-02 23:15:11.996310333 +0000 UTC m=+303.991558626" Jul 02 23:15:22.979014 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:22.978984 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79946c878f-zm7zc" Jul 02 23:15:25.394445 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.394412 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d"] Jul 02 23:15:25.403090 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.403056 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.407021 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.406997 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-55t4w\"" Jul 02 23:15:25.407165 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.407032 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 23:15:25.408334 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.408313 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 23:15:25.416807 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.416777 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d"] Jul 02 23:15:25.536129 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.536088 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56af0311-5e2a-4321-a384-c96be5939e49-tmp\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.536347 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.536159 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56af0311-5e2a-4321-a384-c96be5939e49-tls-certs\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.536347 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.536236 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjvbb\" (UniqueName: \"kubernetes.io/projected/56af0311-5e2a-4321-a384-c96be5939e49-kube-api-access-pjvbb\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.637418 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.637379 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjvbb\" (UniqueName: \"kubernetes.io/projected/56af0311-5e2a-4321-a384-c96be5939e49-kube-api-access-pjvbb\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.637614 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.637459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56af0311-5e2a-4321-a384-c96be5939e49-tmp\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.637614 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.637500 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56af0311-5e2a-4321-a384-c96be5939e49-tls-certs\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.639853 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.639828 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56af0311-5e2a-4321-a384-c96be5939e49-tmp\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.639987 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.639969 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56af0311-5e2a-4321-a384-c96be5939e49-tls-certs\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.653817 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.653758 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjvbb\" (UniqueName: \"kubernetes.io/projected/56af0311-5e2a-4321-a384-c96be5939e49-kube-api-access-pjvbb\") pod \"kube-auth-proxy-86c4d5658d-dgr5d\" (UID: \"56af0311-5e2a-4321-a384-c96be5939e49\") " pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.712994 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.712958 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" Jul 02 23:15:25.860844 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:25.860819 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d"] Jul 02 23:15:25.863253 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:15:25.863221 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56af0311_5e2a_4321_a384_c96be5939e49.slice/crio-86ff46169298e5d2d591dd84d290b521e1b9fe8e0f15eced33fdeb87ff8a0cac WatchSource:0}: Error finding container 86ff46169298e5d2d591dd84d290b521e1b9fe8e0f15eced33fdeb87ff8a0cac: Status 404 returned error can't find the container with id 86ff46169298e5d2d591dd84d290b521e1b9fe8e0f15eced33fdeb87ff8a0cac Jul 02 23:15:26.036868 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.036776 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" event={"ID":"56af0311-5e2a-4321-a384-c96be5939e49","Type":"ContainerStarted","Data":"86ff46169298e5d2d591dd84d290b521e1b9fe8e0f15eced33fdeb87ff8a0cac"} Jul 02 23:15:26.503258 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.503219 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-895fb"] Jul 02 23:15:26.506848 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.506827 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:26.509366 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.509342 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 23:15:26.509503 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.509391 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-9tw69\"" Jul 02 23:15:26.516788 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.516720 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-895fb"] Jul 02 23:15:26.647308 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.647271 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5ns7\" (UniqueName: \"kubernetes.io/projected/1cf74942-3095-410c-a56f-94d5d136515e-kube-api-access-n5ns7\") pod \"model-serving-api-54d89bd5fc-895fb\" (UID: \"1cf74942-3095-410c-a56f-94d5d136515e\") " pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:26.647489 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.647391 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cf74942-3095-410c-a56f-94d5d136515e-tls-certs\") pod \"model-serving-api-54d89bd5fc-895fb\" (UID: \"1cf74942-3095-410c-a56f-94d5d136515e\") " pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:26.748905 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.748833 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cf74942-3095-410c-a56f-94d5d136515e-tls-certs\") pod \"model-serving-api-54d89bd5fc-895fb\" (UID: \"1cf74942-3095-410c-a56f-94d5d136515e\") " pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:26.748905 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.748893 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5ns7\" (UniqueName: \"kubernetes.io/projected/1cf74942-3095-410c-a56f-94d5d136515e-kube-api-access-n5ns7\") pod \"model-serving-api-54d89bd5fc-895fb\" (UID: \"1cf74942-3095-410c-a56f-94d5d136515e\") " pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:26.752637 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.752606 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cf74942-3095-410c-a56f-94d5d136515e-tls-certs\") pod \"model-serving-api-54d89bd5fc-895fb\" (UID: \"1cf74942-3095-410c-a56f-94d5d136515e\") " pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:26.762637 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.762573 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5ns7\" (UniqueName: \"kubernetes.io/projected/1cf74942-3095-410c-a56f-94d5d136515e-kube-api-access-n5ns7\") pod \"model-serving-api-54d89bd5fc-895fb\" (UID: \"1cf74942-3095-410c-a56f-94d5d136515e\") " pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:26.822039 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.821992 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:26.962997 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:26.962963 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-895fb"] Jul 02 23:15:26.966341 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:15:26.966305 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf74942_3095_410c_a56f_94d5d136515e.slice/crio-69a414f2ba4564dba898e10970d65f1c93db3d8f1269d0e2ece3ab17145b7782 WatchSource:0}: Error finding container 69a414f2ba4564dba898e10970d65f1c93db3d8f1269d0e2ece3ab17145b7782: Status 404 returned error can't find the container with id 69a414f2ba4564dba898e10970d65f1c93db3d8f1269d0e2ece3ab17145b7782 Jul 02 23:15:27.043163 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:27.043079 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-895fb" event={"ID":"1cf74942-3095-410c-a56f-94d5d136515e","Type":"ContainerStarted","Data":"69a414f2ba4564dba898e10970d65f1c93db3d8f1269d0e2ece3ab17145b7782"} Jul 02 23:15:28.244236 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.244198 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv"] Jul 02 23:15:28.247643 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.247618 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.254165 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.254015 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 23:15:28.254322 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.254279 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 23:15:28.254322 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.254303 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 23:15:28.254444 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.254373 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-txmbk\"" Jul 02 23:15:28.254444 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.254305 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 23:15:28.254444 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.254390 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 23:15:28.264392 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.264363 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv"] Jul 02 23:15:28.364000 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.363959 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dddb\" (UniqueName: \"kubernetes.io/projected/f6863319-7673-4582-8e9f-d04bf44b3b50-kube-api-access-9dddb\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.364207 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.364019 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6863319-7673-4582-8e9f-d04bf44b3b50-manager-config\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.364207 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.364050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6863319-7673-4582-8e9f-d04bf44b3b50-metrics-cert\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.364207 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.364084 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6863319-7673-4582-8e9f-d04bf44b3b50-cert\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.465472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.465359 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dddb\" (UniqueName: \"kubernetes.io/projected/f6863319-7673-4582-8e9f-d04bf44b3b50-kube-api-access-9dddb\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.465472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.465429 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6863319-7673-4582-8e9f-d04bf44b3b50-manager-config\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.465472 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.465459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6863319-7673-4582-8e9f-d04bf44b3b50-metrics-cert\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.465784 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.465493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6863319-7673-4582-8e9f-d04bf44b3b50-cert\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.466376 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.466325 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6863319-7673-4582-8e9f-d04bf44b3b50-manager-config\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.468906 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.468878 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6863319-7673-4582-8e9f-d04bf44b3b50-cert\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.469035 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.468878 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6863319-7673-4582-8e9f-d04bf44b3b50-metrics-cert\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.474963 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.474913 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dddb\" (UniqueName: \"kubernetes.io/projected/f6863319-7673-4582-8e9f-d04bf44b3b50-kube-api-access-9dddb\") pod \"lws-controller-manager-6898c65679-qfgbv\" (UID: \"f6863319-7673-4582-8e9f-d04bf44b3b50\") " pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:28.559902 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:28.559810 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:29.329159 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:29.329100 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv"] Jul 02 23:15:29.335033 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:15:29.334989 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6863319_7673_4582_8e9f_d04bf44b3b50.slice/crio-e13ccd4da1e6635957a94c2774d51321918c604e38587bfbfa7f6cb7046a546a WatchSource:0}: Error finding container e13ccd4da1e6635957a94c2774d51321918c604e38587bfbfa7f6cb7046a546a: Status 404 returned error can't find the container with id e13ccd4da1e6635957a94c2774d51321918c604e38587bfbfa7f6cb7046a546a Jul 02 23:15:30.061096 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:30.061049 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" event={"ID":"f6863319-7673-4582-8e9f-d04bf44b3b50","Type":"ContainerStarted","Data":"e13ccd4da1e6635957a94c2774d51321918c604e38587bfbfa7f6cb7046a546a"} Jul 02 23:15:30.062766 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:30.062739 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-895fb" event={"ID":"1cf74942-3095-410c-a56f-94d5d136515e","Type":"ContainerStarted","Data":"6e790e13fe1d85fbff2dc1ef1087e139c3c7275525a1207035304d1d1c04d00c"} Jul 02 23:15:30.062893 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:30.062797 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:30.064337 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:30.064318 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" event={"ID":"56af0311-5e2a-4321-a384-c96be5939e49","Type":"ContainerStarted","Data":"04ee79e8f7d887f26474175a342e69fe99ccaedceab245094610b8fe255a53d5"} Jul 02 23:15:30.080458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:30.080397 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-895fb" podStartSLOduration=1.870077443 podStartE2EDuration="4.080375401s" totalImagesPullingTime="2.210297958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:30.079745388 +0000 UTC m=+322.074993707" watchObservedRunningTime="2026-07-02 23:15:30.080375401 +0000 UTC m=+322.075623704" Jul 02 23:15:30.096594 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:30.096545 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86c4d5658d-dgr5d" podStartSLOduration=1.784422909 podStartE2EDuration="5.096528191s" totalImagesPullingTime="3.312105282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:30.095872556 +0000 UTC m=+322.091120865" watchObservedRunningTime="2026-07-02 23:15:30.096528191 +0000 UTC m=+322.091776485" Jul 02 23:15:31.071910 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:31.071871 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" event={"ID":"f6863319-7673-4582-8e9f-d04bf44b3b50","Type":"ContainerStarted","Data":"1e80f23455e857d8f1aa913f71481767fcd259ff9e6db7453726cc6627178da2"} Jul 02 23:15:31.072378 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:31.072040 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:15:31.094248 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:31.094166 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" podStartSLOduration=1.476816563 podStartE2EDuration="3.094147979s" totalImagesPullingTime="1.617331416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:31.093524101 +0000 UTC m=+323.088772383" watchObservedRunningTime="2026-07-02 23:15:31.094147979 +0000 UTC m=+323.089396269" Jul 02 23:15:41.076610 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:41.076576 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-895fb" Jul 02 23:15:42.079009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:15:42.078976 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6898c65679-qfgbv" Jul 02 23:16:42.176975 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.176941 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc"] Jul 02 23:16:42.180005 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.179985 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.182947 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.182924 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:16:42.184173 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.184154 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-txngp\"" Jul 02 23:16:42.184280 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.184161 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:16:42.187178 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.187152 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc"] Jul 02 23:16:42.279230 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.279172 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.279413 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.279238 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j5ng\" (UniqueName: \"kubernetes.io/projected/33dcec3d-240f-4cde-93dc-dcc5436441e5-kube-api-access-2j5ng\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.279413 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.279304 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.380658 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.380621 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.380843 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.380664 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j5ng\" (UniqueName: \"kubernetes.io/projected/33dcec3d-240f-4cde-93dc-dcc5436441e5-kube-api-access-2j5ng\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.380843 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.380746 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.381041 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.381020 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.381111 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.381086 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.389211 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.389171 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j5ng\" (UniqueName: \"kubernetes.io/projected/33dcec3d-240f-4cde-93dc-dcc5436441e5-kube-api-access-2j5ng\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.491353 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.491265 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:42.579131 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.579000 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg"] Jul 02 23:16:42.582007 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.581987 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.598335 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.598272 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg"] Jul 02 23:16:42.631350 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.631318 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc"] Jul 02 23:16:42.634925 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:16:42.634899 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33dcec3d_240f_4cde_93dc_dcc5436441e5.slice/crio-80384c14e4ada452d1f4462430b5acbb5a7658fa07991117d4441db1bca461bb WatchSource:0}: Error finding container 80384c14e4ada452d1f4462430b5acbb5a7658fa07991117d4441db1bca461bb: Status 404 returned error can't find the container with id 80384c14e4ada452d1f4462430b5acbb5a7658fa07991117d4441db1bca461bb Jul 02 23:16:42.682538 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.682469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.682619 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.682535 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.682619 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.682574 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ppmg\" (UniqueName: \"kubernetes.io/projected/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-kube-api-access-5ppmg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.783677 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.783598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.783677 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.783648 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ppmg\" (UniqueName: \"kubernetes.io/projected/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-kube-api-access-5ppmg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.783891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.783716 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.784106 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.784082 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.784176 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.784095 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.792377 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.792354 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ppmg\" (UniqueName: \"kubernetes.io/projected/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-kube-api-access-5ppmg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:42.898107 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:42.898068 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:43.038029 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.037988 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg"] Jul 02 23:16:43.039913 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:16:43.039876 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7c9cda_dad0_414a_9baf_fdae3dfdced0.slice/crio-9b87edf8257a34c0522022a7c209d56eba03707956a573cd86599669b01639f2 WatchSource:0}: Error finding container 9b87edf8257a34c0522022a7c209d56eba03707956a573cd86599669b01639f2: Status 404 returned error can't find the container with id 9b87edf8257a34c0522022a7c209d56eba03707956a573cd86599669b01639f2 Jul 02 23:16:43.174414 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.174380 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k"] Jul 02 23:16:43.177325 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.177303 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.185746 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.185718 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k"] Jul 02 23:16:43.287919 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.287827 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.287919 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.287879 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.287919 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.287906 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9mwk\" (UniqueName: \"kubernetes.io/projected/b3620dde-99de-47d5-a9dd-07b69a05d5d2-kube-api-access-f9mwk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.389103 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.389065 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mwk\" (UniqueName: \"kubernetes.io/projected/b3620dde-99de-47d5-a9dd-07b69a05d5d2-kube-api-access-f9mwk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.389300 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.389247 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.389366 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.389319 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.389617 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.389599 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.389689 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.389637 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.398278 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.398258 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mwk\" (UniqueName: \"kubernetes.io/projected/b3620dde-99de-47d5-a9dd-07b69a05d5d2-kube-api-access-f9mwk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.404953 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.404926 2570 generic.go:356] "Generic (PLEG): container finished" podID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerID="72ef54a0d53a46ff1b9d9fff4ebeccfee5b9640c1c6ef5d2862dba8d06fc7ac2" exitCode=0 Jul 02 23:16:43.405063 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.405003 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" event={"ID":"33dcec3d-240f-4cde-93dc-dcc5436441e5","Type":"ContainerDied","Data":"72ef54a0d53a46ff1b9d9fff4ebeccfee5b9640c1c6ef5d2862dba8d06fc7ac2"} Jul 02 23:16:43.405063 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.405048 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" event={"ID":"33dcec3d-240f-4cde-93dc-dcc5436441e5","Type":"ContainerStarted","Data":"80384c14e4ada452d1f4462430b5acbb5a7658fa07991117d4441db1bca461bb"} Jul 02 23:16:43.406899 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.406880 2570 generic.go:356] "Generic (PLEG): container finished" podID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerID="bdc26d3e2d572ef2af3cd5b25555ef77b34b2560d2a9503ea03fd1b308cc3367" exitCode=0 Jul 02 23:16:43.406983 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.406965 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" event={"ID":"cf7c9cda-dad0-414a-9baf-fdae3dfdced0","Type":"ContainerDied","Data":"bdc26d3e2d572ef2af3cd5b25555ef77b34b2560d2a9503ea03fd1b308cc3367"} Jul 02 23:16:43.407040 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.407006 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" event={"ID":"cf7c9cda-dad0-414a-9baf-fdae3dfdced0","Type":"ContainerStarted","Data":"9b87edf8257a34c0522022a7c209d56eba03707956a573cd86599669b01639f2"} Jul 02 23:16:43.511067 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.511035 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:43.647995 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.647971 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k"] Jul 02 23:16:43.649717 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:16:43.649692 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3620dde_99de_47d5_a9dd_07b69a05d5d2.slice/crio-ed0c2fd25127f784a4f2660abd63886276b86d891ac6d43b08b85894332f43da WatchSource:0}: Error finding container ed0c2fd25127f784a4f2660abd63886276b86d891ac6d43b08b85894332f43da: Status 404 returned error can't find the container with id ed0c2fd25127f784a4f2660abd63886276b86d891ac6d43b08b85894332f43da Jul 02 23:16:43.775496 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.775466 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8"] Jul 02 23:16:43.778645 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.778624 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.787256 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.787227 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8"] Jul 02 23:16:43.792822 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.792798 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.792944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.792847 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.792944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.792909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pzqt\" (UniqueName: \"kubernetes.io/projected/1a584d71-f454-4fa2-b666-2cc56b395866-kube-api-access-9pzqt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.894223 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.894170 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pzqt\" (UniqueName: \"kubernetes.io/projected/1a584d71-f454-4fa2-b666-2cc56b395866-kube-api-access-9pzqt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.894402 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.894244 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.894402 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.894287 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.894723 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.894699 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.894723 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.894713 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:43.905254 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:43.905235 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pzqt\" (UniqueName: \"kubernetes.io/projected/1a584d71-f454-4fa2-b666-2cc56b395866-kube-api-access-9pzqt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:44.116844 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:44.116811 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:44.304788 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:44.304762 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8"] Jul 02 23:16:44.360637 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:16:44.360606 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a584d71_f454_4fa2_b666_2cc56b395866.slice/crio-4b88cb8bfba60f09663235a56a0f2bbcd45200c5718e416c3ec7d8e81fb24559 WatchSource:0}: Error finding container 4b88cb8bfba60f09663235a56a0f2bbcd45200c5718e416c3ec7d8e81fb24559: Status 404 returned error can't find the container with id 4b88cb8bfba60f09663235a56a0f2bbcd45200c5718e416c3ec7d8e81fb24559 Jul 02 23:16:44.415418 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:44.415389 2570 generic.go:356] "Generic (PLEG): container finished" podID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerID="02c468650fa0328d8e15193c678e2f09d8f8476636fa2bfc6669c96e3f918a4d" exitCode=0 Jul 02 23:16:44.415570 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:44.415473 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" event={"ID":"cf7c9cda-dad0-414a-9baf-fdae3dfdced0","Type":"ContainerDied","Data":"02c468650fa0328d8e15193c678e2f09d8f8476636fa2bfc6669c96e3f918a4d"} Jul 02 23:16:44.416690 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:44.416666 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" event={"ID":"1a584d71-f454-4fa2-b666-2cc56b395866","Type":"ContainerStarted","Data":"4b88cb8bfba60f09663235a56a0f2bbcd45200c5718e416c3ec7d8e81fb24559"} Jul 02 23:16:44.418271 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:44.418250 2570 generic.go:356] "Generic (PLEG): container finished" podID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerID="524cf16f9dad3f832af23ae4ba3e7e914a68b49ba4580bd2f75731101a96604b" exitCode=0 Jul 02 23:16:44.418368 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:44.418306 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" event={"ID":"b3620dde-99de-47d5-a9dd-07b69a05d5d2","Type":"ContainerDied","Data":"524cf16f9dad3f832af23ae4ba3e7e914a68b49ba4580bd2f75731101a96604b"} Jul 02 23:16:44.418368 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:44.418336 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" event={"ID":"b3620dde-99de-47d5-a9dd-07b69a05d5d2","Type":"ContainerStarted","Data":"ed0c2fd25127f784a4f2660abd63886276b86d891ac6d43b08b85894332f43da"} Jul 02 23:16:45.424547 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:45.424512 2570 generic.go:356] "Generic (PLEG): container finished" podID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerID="4a4ca0ef63c9e4bf6e06ac5d17391d6784ba1442b2772d65d3258c26da4e5f8c" exitCode=0 Jul 02 23:16:45.424959 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:45.424604 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" event={"ID":"33dcec3d-240f-4cde-93dc-dcc5436441e5","Type":"ContainerDied","Data":"4a4ca0ef63c9e4bf6e06ac5d17391d6784ba1442b2772d65d3258c26da4e5f8c"} Jul 02 23:16:45.426986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:45.426964 2570 generic.go:356] "Generic (PLEG): container finished" podID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerID="c1a167a434596d78f236f34723fbf6ee7ab77bcb8b393dfbadb43a5524eb86a0" exitCode=0 Jul 02 23:16:45.427144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:45.427041 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" event={"ID":"cf7c9cda-dad0-414a-9baf-fdae3dfdced0","Type":"ContainerDied","Data":"c1a167a434596d78f236f34723fbf6ee7ab77bcb8b393dfbadb43a5524eb86a0"} Jul 02 23:16:45.428586 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:45.428560 2570 generic.go:356] "Generic (PLEG): container finished" podID="1a584d71-f454-4fa2-b666-2cc56b395866" containerID="d8105808da38dbe58180197ec4447e5a9197e35f680ad146f5daa8219ac1f15b" exitCode=0 Jul 02 23:16:45.428685 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:45.428598 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" event={"ID":"1a584d71-f454-4fa2-b666-2cc56b395866","Type":"ContainerDied","Data":"d8105808da38dbe58180197ec4447e5a9197e35f680ad146f5daa8219ac1f15b"} Jul 02 23:16:45.430638 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:45.430566 2570 generic.go:356] "Generic (PLEG): container finished" podID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerID="f64af5666fb04ccf156c2705299b2d8c55830fc7b94629b9c047d81644c2ba34" exitCode=0 Jul 02 23:16:45.430638 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:45.430624 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" event={"ID":"b3620dde-99de-47d5-a9dd-07b69a05d5d2","Type":"ContainerDied","Data":"f64af5666fb04ccf156c2705299b2d8c55830fc7b94629b9c047d81644c2ba34"} Jul 02 23:16:46.438224 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.438111 2570 generic.go:356] "Generic (PLEG): container finished" podID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerID="5ed36afefa8501080ff01dd2cbb026c15e43ccf52f339a29b3c9ccc6e987884d" exitCode=0 Jul 02 23:16:46.438619 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.438213 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" event={"ID":"b3620dde-99de-47d5-a9dd-07b69a05d5d2","Type":"ContainerDied","Data":"5ed36afefa8501080ff01dd2cbb026c15e43ccf52f339a29b3c9ccc6e987884d"} Jul 02 23:16:46.440198 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.440162 2570 generic.go:356] "Generic (PLEG): container finished" podID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerID="0897d743cf8221d6132a538b0cf0b985204a7aa80424f594900de3b1da255ef6" exitCode=0 Jul 02 23:16:46.440300 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.440210 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" event={"ID":"33dcec3d-240f-4cde-93dc-dcc5436441e5","Type":"ContainerDied","Data":"0897d743cf8221d6132a538b0cf0b985204a7aa80424f594900de3b1da255ef6"} Jul 02 23:16:46.441806 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.441788 2570 generic.go:356] "Generic (PLEG): container finished" podID="1a584d71-f454-4fa2-b666-2cc56b395866" containerID="053228419f8d3c4f106d9164da2673bbf644b35026819603bf87d3b9cf4f955f" exitCode=0 Jul 02 23:16:46.441916 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.441871 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" event={"ID":"1a584d71-f454-4fa2-b666-2cc56b395866","Type":"ContainerDied","Data":"053228419f8d3c4f106d9164da2673bbf644b35026819603bf87d3b9cf4f955f"} Jul 02 23:16:46.579013 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.578984 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:46.621713 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.621674 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-util\") pod \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " Jul 02 23:16:46.621895 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.621756 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-bundle\") pod \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " Jul 02 23:16:46.621895 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.621796 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ppmg\" (UniqueName: \"kubernetes.io/projected/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-kube-api-access-5ppmg\") pod \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\" (UID: \"cf7c9cda-dad0-414a-9baf-fdae3dfdced0\") " Jul 02 23:16:46.622336 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.622306 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-bundle" (OuterVolumeSpecName: "bundle") pod "cf7c9cda-dad0-414a-9baf-fdae3dfdced0" (UID: "cf7c9cda-dad0-414a-9baf-fdae3dfdced0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:16:46.624100 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.624078 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-kube-api-access-5ppmg" (OuterVolumeSpecName: "kube-api-access-5ppmg") pod "cf7c9cda-dad0-414a-9baf-fdae3dfdced0" (UID: "cf7c9cda-dad0-414a-9baf-fdae3dfdced0"). InnerVolumeSpecName "kube-api-access-5ppmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:16:46.627153 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.627120 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-util" (OuterVolumeSpecName: "util") pod "cf7c9cda-dad0-414a-9baf-fdae3dfdced0" (UID: "cf7c9cda-dad0-414a-9baf-fdae3dfdced0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:16:46.722486 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.722402 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ppmg\" (UniqueName: \"kubernetes.io/projected/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-kube-api-access-5ppmg\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:46.722486 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.722430 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-util\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:46.722486 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:46.722440 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf7c9cda-dad0-414a-9baf-fdae3dfdced0-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:47.452754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.452726 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" Jul 02 23:16:47.453170 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.452716 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg" event={"ID":"cf7c9cda-dad0-414a-9baf-fdae3dfdced0","Type":"ContainerDied","Data":"9b87edf8257a34c0522022a7c209d56eba03707956a573cd86599669b01639f2"} Jul 02 23:16:47.453170 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.452838 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b87edf8257a34c0522022a7c209d56eba03707956a573cd86599669b01639f2" Jul 02 23:16:47.454836 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.454810 2570 generic.go:356] "Generic (PLEG): container finished" podID="1a584d71-f454-4fa2-b666-2cc56b395866" containerID="013a8a261d2ad9c8aab05d9da039e1d938965b8459467fbb03cf477a7d7c4e2b" exitCode=0 Jul 02 23:16:47.454974 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.454894 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" event={"ID":"1a584d71-f454-4fa2-b666-2cc56b395866","Type":"ContainerDied","Data":"013a8a261d2ad9c8aab05d9da039e1d938965b8459467fbb03cf477a7d7c4e2b"} Jul 02 23:16:47.611563 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.611530 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:47.613617 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.613592 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:47.729864 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.729777 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-util\") pod \"33dcec3d-240f-4cde-93dc-dcc5436441e5\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " Jul 02 23:16:47.729864 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.729828 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-bundle\") pod \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " Jul 02 23:16:47.729864 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.729863 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j5ng\" (UniqueName: \"kubernetes.io/projected/33dcec3d-240f-4cde-93dc-dcc5436441e5-kube-api-access-2j5ng\") pod \"33dcec3d-240f-4cde-93dc-dcc5436441e5\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " Jul 02 23:16:47.730156 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.729934 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-bundle\") pod \"33dcec3d-240f-4cde-93dc-dcc5436441e5\" (UID: \"33dcec3d-240f-4cde-93dc-dcc5436441e5\") " Jul 02 23:16:47.730156 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.729960 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-util\") pod \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " Jul 02 23:16:47.730156 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.730075 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9mwk\" (UniqueName: \"kubernetes.io/projected/b3620dde-99de-47d5-a9dd-07b69a05d5d2-kube-api-access-f9mwk\") pod \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\" (UID: \"b3620dde-99de-47d5-a9dd-07b69a05d5d2\") " Jul 02 23:16:47.730495 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.730465 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-bundle" (OuterVolumeSpecName: "bundle") pod "33dcec3d-240f-4cde-93dc-dcc5436441e5" (UID: "33dcec3d-240f-4cde-93dc-dcc5436441e5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:16:47.730628 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.730471 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-bundle" (OuterVolumeSpecName: "bundle") pod "b3620dde-99de-47d5-a9dd-07b69a05d5d2" (UID: "b3620dde-99de-47d5-a9dd-07b69a05d5d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:16:47.732486 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.732456 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33dcec3d-240f-4cde-93dc-dcc5436441e5-kube-api-access-2j5ng" (OuterVolumeSpecName: "kube-api-access-2j5ng") pod "33dcec3d-240f-4cde-93dc-dcc5436441e5" (UID: "33dcec3d-240f-4cde-93dc-dcc5436441e5"). InnerVolumeSpecName "kube-api-access-2j5ng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:16:47.732615 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.732596 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3620dde-99de-47d5-a9dd-07b69a05d5d2-kube-api-access-f9mwk" (OuterVolumeSpecName: "kube-api-access-f9mwk") pod "b3620dde-99de-47d5-a9dd-07b69a05d5d2" (UID: "b3620dde-99de-47d5-a9dd-07b69a05d5d2"). InnerVolumeSpecName "kube-api-access-f9mwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:16:47.735533 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.735513 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-util" (OuterVolumeSpecName: "util") pod "33dcec3d-240f-4cde-93dc-dcc5436441e5" (UID: "33dcec3d-240f-4cde-93dc-dcc5436441e5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:16:47.736387 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.736368 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-util" (OuterVolumeSpecName: "util") pod "b3620dde-99de-47d5-a9dd-07b69a05d5d2" (UID: "b3620dde-99de-47d5-a9dd-07b69a05d5d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:16:47.831754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.831717 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:47.831754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.831748 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-util\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:47.831754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.831758 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9mwk\" (UniqueName: \"kubernetes.io/projected/b3620dde-99de-47d5-a9dd-07b69a05d5d2-kube-api-access-f9mwk\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:47.831915 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.831767 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/33dcec3d-240f-4cde-93dc-dcc5436441e5-util\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:47.831915 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.831776 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3620dde-99de-47d5-a9dd-07b69a05d5d2-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:47.831915 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:47.831785 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j5ng\" (UniqueName: \"kubernetes.io/projected/33dcec3d-240f-4cde-93dc-dcc5436441e5-kube-api-access-2j5ng\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:48.461581 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.461556 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" Jul 02 23:16:48.462056 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.461583 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc" event={"ID":"33dcec3d-240f-4cde-93dc-dcc5436441e5","Type":"ContainerDied","Data":"80384c14e4ada452d1f4462430b5acbb5a7658fa07991117d4441db1bca461bb"} Jul 02 23:16:48.462056 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.461630 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80384c14e4ada452d1f4462430b5acbb5a7658fa07991117d4441db1bca461bb" Jul 02 23:16:48.463526 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.463501 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" Jul 02 23:16:48.463628 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.463523 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k" event={"ID":"b3620dde-99de-47d5-a9dd-07b69a05d5d2","Type":"ContainerDied","Data":"ed0c2fd25127f784a4f2660abd63886276b86d891ac6d43b08b85894332f43da"} Jul 02 23:16:48.463628 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.463560 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed0c2fd25127f784a4f2660abd63886276b86d891ac6d43b08b85894332f43da" Jul 02 23:16:48.596133 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.596096 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:48.640001 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.639958 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-bundle\") pod \"1a584d71-f454-4fa2-b666-2cc56b395866\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " Jul 02 23:16:48.640171 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.640025 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pzqt\" (UniqueName: \"kubernetes.io/projected/1a584d71-f454-4fa2-b666-2cc56b395866-kube-api-access-9pzqt\") pod \"1a584d71-f454-4fa2-b666-2cc56b395866\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " Jul 02 23:16:48.640171 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.640058 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-util\") pod \"1a584d71-f454-4fa2-b666-2cc56b395866\" (UID: \"1a584d71-f454-4fa2-b666-2cc56b395866\") " Jul 02 23:16:48.640763 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.640732 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-bundle" (OuterVolumeSpecName: "bundle") pod "1a584d71-f454-4fa2-b666-2cc56b395866" (UID: "1a584d71-f454-4fa2-b666-2cc56b395866"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:16:48.642495 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.642398 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a584d71-f454-4fa2-b666-2cc56b395866-kube-api-access-9pzqt" (OuterVolumeSpecName: "kube-api-access-9pzqt") pod "1a584d71-f454-4fa2-b666-2cc56b395866" (UID: "1a584d71-f454-4fa2-b666-2cc56b395866"). InnerVolumeSpecName "kube-api-access-9pzqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:16:48.645998 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.645975 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-util" (OuterVolumeSpecName: "util") pod "1a584d71-f454-4fa2-b666-2cc56b395866" (UID: "1a584d71-f454-4fa2-b666-2cc56b395866"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:16:48.741968 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.741876 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:48.741968 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.741905 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pzqt\" (UniqueName: \"kubernetes.io/projected/1a584d71-f454-4fa2-b666-2cc56b395866-kube-api-access-9pzqt\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:48.741968 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:48.741915 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a584d71-f454-4fa2-b666-2cc56b395866-util\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:16:49.470060 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:49.470032 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" Jul 02 23:16:49.470460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:49.470057 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8" event={"ID":"1a584d71-f454-4fa2-b666-2cc56b395866","Type":"ContainerDied","Data":"4b88cb8bfba60f09663235a56a0f2bbcd45200c5718e416c3ec7d8e81fb24559"} Jul 02 23:16:49.470460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:16:49.470101 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b88cb8bfba60f09663235a56a0f2bbcd45200c5718e416c3ec7d8e81fb24559" Jul 02 23:17:00.888044 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888012 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78657dfb95-f8cvm"] Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888334 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerName="util" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888347 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerName="util" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888359 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerName="pull" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888365 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerName="pull" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888371 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a584d71-f454-4fa2-b666-2cc56b395866" containerName="util" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888377 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a584d71-f454-4fa2-b666-2cc56b395866" containerName="util" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888384 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerName="pull" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888390 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerName="pull" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888396 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerName="extract" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888401 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerName="extract" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888408 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerName="util" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888412 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerName="util" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888420 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a584d71-f454-4fa2-b666-2cc56b395866" containerName="pull" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888426 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a584d71-f454-4fa2-b666-2cc56b395866" containerName="pull" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888435 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerName="pull" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888441 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerName="pull" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888447 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerName="extract" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888452 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerName="extract" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888460 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a584d71-f454-4fa2-b666-2cc56b395866" containerName="extract" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888465 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a584d71-f454-4fa2-b666-2cc56b395866" containerName="extract" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888471 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerName="util" Jul 02 23:17:00.888458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888476 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerName="util" Jul 02 23:17:00.889162 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888483 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerName="extract" Jul 02 23:17:00.889162 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888490 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerName="extract" Jul 02 23:17:00.889162 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888550 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="33dcec3d-240f-4cde-93dc-dcc5436441e5" containerName="extract" Jul 02 23:17:00.889162 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888557 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3620dde-99de-47d5-a9dd-07b69a05d5d2" containerName="extract" Jul 02 23:17:00.889162 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888563 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf7c9cda-dad0-414a-9baf-fdae3dfdced0" containerName="extract" Jul 02 23:17:00.889162 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.888571 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a584d71-f454-4fa2-b666-2cc56b395866" containerName="extract" Jul 02 23:17:00.891791 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.891773 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:00.902467 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.902441 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78657dfb95-f8cvm"] Jul 02 23:17:00.948358 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.948323 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv8pt\" (UniqueName: \"kubernetes.io/projected/075561db-ccca-4595-9dad-454ed29408e9-kube-api-access-pv8pt\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:00.948358 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.948367 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-oauth-serving-cert\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:00.948541 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.948388 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/075561db-ccca-4595-9dad-454ed29408e9-console-oauth-config\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:00.948541 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.948469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-trusted-ca-bundle\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:00.948631 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.948545 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-service-ca\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:00.948631 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.948569 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-console-config\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:00.948631 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:00.948589 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/075561db-ccca-4595-9dad-454ed29408e9-console-serving-cert\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.049457 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.049419 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv8pt\" (UniqueName: \"kubernetes.io/projected/075561db-ccca-4595-9dad-454ed29408e9-kube-api-access-pv8pt\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.049457 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.049458 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-oauth-serving-cert\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.049710 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.049478 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/075561db-ccca-4595-9dad-454ed29408e9-console-oauth-config\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.049710 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.049509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-trusted-ca-bundle\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.049710 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.049551 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-service-ca\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.049710 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.049572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-console-config\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.049710 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.049594 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/075561db-ccca-4595-9dad-454ed29408e9-console-serving-cert\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.050403 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.050367 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-service-ca\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.050521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.050367 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-oauth-serving-cert\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.050521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.050437 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-trusted-ca-bundle\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.050521 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.050453 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/075561db-ccca-4595-9dad-454ed29408e9-console-config\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.051996 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.051969 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/075561db-ccca-4595-9dad-454ed29408e9-console-serving-cert\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.052104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.052060 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/075561db-ccca-4595-9dad-454ed29408e9-console-oauth-config\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.057634 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.057610 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv8pt\" (UniqueName: \"kubernetes.io/projected/075561db-ccca-4595-9dad-454ed29408e9-kube-api-access-pv8pt\") pod \"console-78657dfb95-f8cvm\" (UID: \"075561db-ccca-4595-9dad-454ed29408e9\") " pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.201969 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.201867 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:01.338769 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.338738 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78657dfb95-f8cvm"] Jul 02 23:17:01.340413 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:17:01.340387 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075561db_ccca_4595_9dad_454ed29408e9.slice/crio-07a4b63a2c0c94777478a210013f50e271ffec96a50534cdb1c51f2beca6c406 WatchSource:0}: Error finding container 07a4b63a2c0c94777478a210013f50e271ffec96a50534cdb1c51f2beca6c406: Status 404 returned error can't find the container with id 07a4b63a2c0c94777478a210013f50e271ffec96a50534cdb1c51f2beca6c406 Jul 02 23:17:01.529613 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.529534 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78657dfb95-f8cvm" event={"ID":"075561db-ccca-4595-9dad-454ed29408e9","Type":"ContainerStarted","Data":"5886f583a9e89b18582ed7e79840bcb09685c724c853e1e6d54e843ea0ff7ba5"} Jul 02 23:17:01.529613 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.529579 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78657dfb95-f8cvm" event={"ID":"075561db-ccca-4595-9dad-454ed29408e9","Type":"ContainerStarted","Data":"07a4b63a2c0c94777478a210013f50e271ffec96a50534cdb1c51f2beca6c406"} Jul 02 23:17:01.550256 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:01.550204 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78657dfb95-f8cvm" podStartSLOduration=1.55017448 podStartE2EDuration="1.55017448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:01.548694216 +0000 UTC m=+413.543942507" watchObservedRunningTime="2026-07-02 23:17:01.55017448 +0000 UTC m=+413.545422793" Jul 02 23:17:11.202984 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:11.202942 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:11.203419 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:11.203037 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:11.207735 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:11.207710 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:11.581004 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:11.580979 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78657dfb95-f8cvm" Jul 02 23:17:11.638272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:11.638158 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-666fdb9888-h4qv4"] Jul 02 23:17:36.660327 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.660214 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-666fdb9888-h4qv4" podUID="c8ac7e55-f8e3-4f64-a993-3482cea0472d" containerName="console" containerID="cri-o://fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855" gracePeriod=15 Jul 02 23:17:36.917881 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.917818 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:17:36.943415 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943382 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-oauth-config\") pod \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " Jul 02 23:17:36.943415 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943415 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-oauth-serving-cert\") pod \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " Jul 02 23:17:36.943671 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943436 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htpw5\" (UniqueName: \"kubernetes.io/projected/c8ac7e55-f8e3-4f64-a993-3482cea0472d-kube-api-access-htpw5\") pod \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " Jul 02 23:17:36.943671 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943457 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-trusted-ca-bundle\") pod \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " Jul 02 23:17:36.943671 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943492 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-serving-cert\") pod \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " Jul 02 23:17:36.943671 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943543 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-config\") pod \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " Jul 02 23:17:36.943671 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943569 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-service-ca\") pod \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\" (UID: \"c8ac7e55-f8e3-4f64-a993-3482cea0472d\") " Jul 02 23:17:36.943999 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943857 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c8ac7e55-f8e3-4f64-a993-3482cea0472d" (UID: "c8ac7e55-f8e3-4f64-a993-3482cea0472d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:17:36.943999 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.943883 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c8ac7e55-f8e3-4f64-a993-3482cea0472d" (UID: "c8ac7e55-f8e3-4f64-a993-3482cea0472d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:17:36.944303 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.944260 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-service-ca" (OuterVolumeSpecName: "service-ca") pod "c8ac7e55-f8e3-4f64-a993-3482cea0472d" (UID: "c8ac7e55-f8e3-4f64-a993-3482cea0472d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:17:36.946102 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.944354 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-config" (OuterVolumeSpecName: "console-config") pod "c8ac7e55-f8e3-4f64-a993-3482cea0472d" (UID: "c8ac7e55-f8e3-4f64-a993-3482cea0472d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 23:17:36.946387 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.946351 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c8ac7e55-f8e3-4f64-a993-3482cea0472d" (UID: "c8ac7e55-f8e3-4f64-a993-3482cea0472d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:17:36.946484 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.946453 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ac7e55-f8e3-4f64-a993-3482cea0472d-kube-api-access-htpw5" (OuterVolumeSpecName: "kube-api-access-htpw5") pod "c8ac7e55-f8e3-4f64-a993-3482cea0472d" (UID: "c8ac7e55-f8e3-4f64-a993-3482cea0472d"). InnerVolumeSpecName "kube-api-access-htpw5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:17:36.946586 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:36.946564 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c8ac7e55-f8e3-4f64-a993-3482cea0472d" (UID: "c8ac7e55-f8e3-4f64-a993-3482cea0472d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:17:37.044650 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.044601 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-oauth-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:17:37.044650 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.044641 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-oauth-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:17:37.044650 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.044651 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htpw5\" (UniqueName: \"kubernetes.io/projected/c8ac7e55-f8e3-4f64-a993-3482cea0472d-kube-api-access-htpw5\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:17:37.044650 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.044660 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-trusted-ca-bundle\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:17:37.044650 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.044669 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-serving-cert\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:17:37.044953 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.044677 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-console-config\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:17:37.044953 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.044688 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ac7e55-f8e3-4f64-a993-3482cea0472d-service-ca\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:17:37.713176 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.713139 2570 generic.go:356] "Generic (PLEG): container finished" podID="c8ac7e55-f8e3-4f64-a993-3482cea0472d" containerID="fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855" exitCode=0 Jul 02 23:17:37.713176 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.713176 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-666fdb9888-h4qv4" Jul 02 23:17:37.713694 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.713214 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666fdb9888-h4qv4" event={"ID":"c8ac7e55-f8e3-4f64-a993-3482cea0472d","Type":"ContainerDied","Data":"fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855"} Jul 02 23:17:37.713694 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.713269 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-666fdb9888-h4qv4" event={"ID":"c8ac7e55-f8e3-4f64-a993-3482cea0472d","Type":"ContainerDied","Data":"7a4e6836cffa285c6ee278312aa261dc44741c932d911e256f40e312a4d1bccf"} Jul 02 23:17:37.713694 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.713298 2570 scope.go:117] "RemoveContainer" containerID="fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855" Jul 02 23:17:37.722738 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.722721 2570 scope.go:117] "RemoveContainer" containerID="fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855" Jul 02 23:17:37.722995 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:17:37.722975 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855\": container with ID starting with fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855 not found: ID does not exist" containerID="fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855" Jul 02 23:17:37.723054 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.723008 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855"} err="failed to get container status \"fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855\": rpc error: code = NotFound desc = could not find container \"fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855\": container with ID starting with fb6b3548f28568a2426752ebf40c08ba65a96ec98c564b6ebbb87512f1ea2855 not found: ID does not exist" Jul 02 23:17:37.744107 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.744073 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-666fdb9888-h4qv4"] Jul 02 23:17:37.753710 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:37.753680 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-666fdb9888-h4qv4"] Jul 02 23:17:38.592870 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:17:38.592826 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ac7e55-f8e3-4f64-a993-3482cea0472d" path="/var/lib/kubelet/pods/c8ac7e55-f8e3-4f64-a993-3482cea0472d/volumes" Jul 02 23:18:05.327296 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.327252 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:18:05.327755 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.327722 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ac7e55-f8e3-4f64-a993-3482cea0472d" containerName="console" Jul 02 23:18:05.327755 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.327740 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ac7e55-f8e3-4f64-a993-3482cea0472d" containerName="console" Jul 02 23:18:05.327871 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.327850 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ac7e55-f8e3-4f64-a993-3482cea0472d" containerName="console" Jul 02 23:18:05.330803 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.330780 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:05.333517 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.333492 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 23:18:05.333931 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.333915 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 23:18:05.334798 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.334778 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 23:18:05.334856 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.334782 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-txngp\"" Jul 02 23:18:05.349884 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.349836 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:18:05.366621 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.366591 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:18:05.392078 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.392043 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5t5m\" (UniqueName: \"kubernetes.io/projected/4d5fabed-73af-4192-8ca6-ec1b0d566d95-kube-api-access-r5t5m\") pod \"limitador-limitador-54d6d988d5-cjfdn\" (UID: \"4d5fabed-73af-4192-8ca6-ec1b0d566d95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:05.392078 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.392085 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d5fabed-73af-4192-8ca6-ec1b0d566d95-config-file\") pod \"limitador-limitador-54d6d988d5-cjfdn\" (UID: \"4d5fabed-73af-4192-8ca6-ec1b0d566d95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:05.493340 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.493301 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5t5m\" (UniqueName: \"kubernetes.io/projected/4d5fabed-73af-4192-8ca6-ec1b0d566d95-kube-api-access-r5t5m\") pod \"limitador-limitador-54d6d988d5-cjfdn\" (UID: \"4d5fabed-73af-4192-8ca6-ec1b0d566d95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:05.493522 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.493350 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d5fabed-73af-4192-8ca6-ec1b0d566d95-config-file\") pod \"limitador-limitador-54d6d988d5-cjfdn\" (UID: \"4d5fabed-73af-4192-8ca6-ec1b0d566d95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:05.494065 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.494040 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d5fabed-73af-4192-8ca6-ec1b0d566d95-config-file\") pod \"limitador-limitador-54d6d988d5-cjfdn\" (UID: \"4d5fabed-73af-4192-8ca6-ec1b0d566d95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:05.506466 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.506436 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5t5m\" (UniqueName: \"kubernetes.io/projected/4d5fabed-73af-4192-8ca6-ec1b0d566d95-kube-api-access-r5t5m\") pod \"limitador-limitador-54d6d988d5-cjfdn\" (UID: \"4d5fabed-73af-4192-8ca6-ec1b0d566d95\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:05.641327 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.641234 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:05.784775 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.784721 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:18:05.788634 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:18:05.788600 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d5fabed_73af_4192_8ca6_ec1b0d566d95.slice/crio-bb68751ef04ae317bf51628ffb908a27a64ffb502d1c40d6f7959a3fb9edb299 WatchSource:0}: Error finding container bb68751ef04ae317bf51628ffb908a27a64ffb502d1c40d6f7959a3fb9edb299: Status 404 returned error can't find the container with id bb68751ef04ae317bf51628ffb908a27a64ffb502d1c40d6f7959a3fb9edb299 Jul 02 23:18:05.862392 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:05.862351 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" event={"ID":"4d5fabed-73af-4192-8ca6-ec1b0d566d95","Type":"ContainerStarted","Data":"bb68751ef04ae317bf51628ffb908a27a64ffb502d1c40d6f7959a3fb9edb299"} Jul 02 23:18:06.032114 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.032036 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6nx2"] Jul 02 23:18:06.036339 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.036319 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p6nx2" Jul 02 23:18:06.040080 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.040056 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bbmch\"" Jul 02 23:18:06.041396 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.041375 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6nx2"] Jul 02 23:18:06.098458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.098422 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw4tx\" (UniqueName: \"kubernetes.io/projected/6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b-kube-api-access-nw4tx\") pod \"authorino-75d556bd77-p6nx2\" (UID: \"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b\") " pod="kuadrant-system/authorino-75d556bd77-p6nx2" Jul 02 23:18:06.199112 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.199078 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw4tx\" (UniqueName: \"kubernetes.io/projected/6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b-kube-api-access-nw4tx\") pod \"authorino-75d556bd77-p6nx2\" (UID: \"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b\") " pod="kuadrant-system/authorino-75d556bd77-p6nx2" Jul 02 23:18:06.208632 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.208605 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw4tx\" (UniqueName: \"kubernetes.io/projected/6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b-kube-api-access-nw4tx\") pod \"authorino-75d556bd77-p6nx2\" (UID: \"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b\") " pod="kuadrant-system/authorino-75d556bd77-p6nx2" Jul 02 23:18:06.346683 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.346649 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p6nx2" Jul 02 23:18:06.534042 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.533971 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6nx2"] Jul 02 23:18:06.541519 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:18:06.541475 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d3e8aeb_e0f9_4958_b24a_11d3e5718d3b.slice/crio-11a7995a19bc82266589eb29d4b6d713ba8463f86dddcac2de78b2fa30bc940c WatchSource:0}: Error finding container 11a7995a19bc82266589eb29d4b6d713ba8463f86dddcac2de78b2fa30bc940c: Status 404 returned error can't find the container with id 11a7995a19bc82266589eb29d4b6d713ba8463f86dddcac2de78b2fa30bc940c Jul 02 23:18:06.870692 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:06.870646 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p6nx2" event={"ID":"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b","Type":"ContainerStarted","Data":"11a7995a19bc82266589eb29d4b6d713ba8463f86dddcac2de78b2fa30bc940c"} Jul 02 23:18:10.900944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:10.900908 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" event={"ID":"4d5fabed-73af-4192-8ca6-ec1b0d566d95","Type":"ContainerStarted","Data":"c63f303a67dedecfda09facab49bd86c22263147411e1748225f16038a60a749"} Jul 02 23:18:10.901471 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:10.901030 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:10.902570 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:10.902547 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p6nx2" event={"ID":"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b","Type":"ContainerStarted","Data":"48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e"} Jul 02 23:18:10.919545 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:10.919486 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" podStartSLOduration=1.408567843 podStartE2EDuration="5.919471631s" totalImagesPullingTime="4.510903788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:10.91794708 +0000 UTC m=+482.913195371" watchObservedRunningTime="2026-07-02 23:18:10.919471631 +0000 UTC m=+482.914719921" Jul 02 23:18:10.935114 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:10.935067 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-p6nx2" podStartSLOduration=1.172376279 podStartE2EDuration="4.935049883s" totalImagesPullingTime="3.762673604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:10.9340897 +0000 UTC m=+482.929337992" watchObservedRunningTime="2026-07-02 23:18:10.935049883 +0000 UTC m=+482.930298177" Jul 02 23:18:21.909674 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:21.909637 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cjfdn" Jul 02 23:18:40.006652 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:40.006610 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6nx2"] Jul 02 23:18:40.007236 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:40.006782 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-p6nx2" podUID="6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b" containerName="authorino" containerID="cri-o://48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e" gracePeriod=30 Jul 02 23:18:40.263580 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:40.263520 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p6nx2" Jul 02 23:18:40.266492 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:40.266470 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw4tx\" (UniqueName: \"kubernetes.io/projected/6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b-kube-api-access-nw4tx\") pod \"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b\" (UID: \"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b\") " Jul 02 23:18:40.268406 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:40.268376 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b-kube-api-access-nw4tx" (OuterVolumeSpecName: "kube-api-access-nw4tx") pod "6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b" (UID: "6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b"). InnerVolumeSpecName "kube-api-access-nw4tx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:18:40.367173 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:40.367136 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw4tx\" (UniqueName: \"kubernetes.io/projected/6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b-kube-api-access-nw4tx\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:18:41.060137 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.060049 2570 generic.go:356] "Generic (PLEG): container finished" podID="6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b" containerID="48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e" exitCode=0 Jul 02 23:18:41.060137 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.060095 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p6nx2" Jul 02 23:18:41.060137 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.060101 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p6nx2" event={"ID":"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b","Type":"ContainerDied","Data":"48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e"} Jul 02 23:18:41.060137 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.060137 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p6nx2" event={"ID":"6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b","Type":"ContainerDied","Data":"11a7995a19bc82266589eb29d4b6d713ba8463f86dddcac2de78b2fa30bc940c"} Jul 02 23:18:41.060702 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.060162 2570 scope.go:117] "RemoveContainer" containerID="48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e" Jul 02 23:18:41.068681 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.068650 2570 scope.go:117] "RemoveContainer" containerID="48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e" Jul 02 23:18:41.069042 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:18:41.069015 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e\": container with ID starting with 48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e not found: ID does not exist" containerID="48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e" Jul 02 23:18:41.069092 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.069061 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e"} err="failed to get container status \"48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e\": rpc error: code = NotFound desc = could not find container \"48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e\": container with ID starting with 48965627d4d634c51402bb5d0a206a91c468caf8bc8747c02b2814b26f9fb41e not found: ID does not exist" Jul 02 23:18:41.077323 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.077294 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6nx2"] Jul 02 23:18:41.080144 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:41.080121 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6nx2"] Jul 02 23:18:42.592448 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:18:42.592413 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b" path="/var/lib/kubelet/pods/6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b/volumes" Jul 02 23:19:21.945311 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:19:21.945272 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:19:39.281111 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:19:39.281077 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:19:49.802601 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:19:49.802562 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:20:09.031236 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:09.031109 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:20:09.033084 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:09.032655 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:20:09.040774 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:09.040750 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:20:09.042801 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:09.042780 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:20:10.494090 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:10.494053 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:20:20.492609 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:20.492574 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:20:22.268891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.268853 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr"] Jul 02 23:20:22.269278 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.269213 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b" containerName="authorino" Jul 02 23:20:22.269278 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.269225 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b" containerName="authorino" Jul 02 23:20:22.269358 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.269291 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d3e8aeb-e0f9-4958-b24a-11d3e5718d3b" containerName="authorino" Jul 02 23:20:22.271388 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.271368 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.275488 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.275464 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 23:20:22.275488 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.275487 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 02 23:20:22.275675 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.275463 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-djwms\"" Jul 02 23:20:22.275675 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.275464 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 23:20:22.280796 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.280773 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr"] Jul 02 23:20:22.388109 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.388068 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.388109 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.388114 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.388330 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.388142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/162add50-38b2-4d51-b8cf-f665a608dd7b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.388330 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.388251 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.388330 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.388287 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.388429 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.388381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6f4x\" (UniqueName: \"kubernetes.io/projected/162add50-38b2-4d51-b8cf-f665a608dd7b-kube-api-access-g6f4x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.488862 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.488819 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6f4x\" (UniqueName: \"kubernetes.io/projected/162add50-38b2-4d51-b8cf-f665a608dd7b-kube-api-access-g6f4x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.489062 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.488884 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.489062 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.488924 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.489062 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.488948 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/162add50-38b2-4d51-b8cf-f665a608dd7b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.489062 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.488985 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.489062 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.489010 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.489382 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.489317 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.489435 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.489384 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.489557 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.489531 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.491216 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.491196 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/162add50-38b2-4d51-b8cf-f665a608dd7b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.491490 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.491472 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/162add50-38b2-4d51-b8cf-f665a608dd7b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.497070 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.497043 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6f4x\" (UniqueName: \"kubernetes.io/projected/162add50-38b2-4d51-b8cf-f665a608dd7b-kube-api-access-g6f4x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-xs4hr\" (UID: \"162add50-38b2-4d51-b8cf-f665a608dd7b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.583110 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.583072 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:22.719448 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.719418 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr"] Jul 02 23:20:22.721352 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:20:22.721316 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod162add50_38b2_4d51_b8cf_f665a608dd7b.slice/crio-03b5b9353bc8e0831d472ebb0118fb2b5c75f74085d3e44676482232132cbf4f WatchSource:0}: Error finding container 03b5b9353bc8e0831d472ebb0118fb2b5c75f74085d3e44676482232132cbf4f: Status 404 returned error can't find the container with id 03b5b9353bc8e0831d472ebb0118fb2b5c75f74085d3e44676482232132cbf4f Jul 02 23:20:22.722622 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:22.722603 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:20:23.568025 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:23.567990 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" event={"ID":"162add50-38b2-4d51-b8cf-f665a608dd7b","Type":"ContainerStarted","Data":"03b5b9353bc8e0831d472ebb0118fb2b5c75f74085d3e44676482232132cbf4f"} Jul 02 23:20:24.574536 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:24.574497 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" event={"ID":"162add50-38b2-4d51-b8cf-f665a608dd7b","Type":"ContainerStarted","Data":"451dd62bc5ec6f2f2b0bb2fc9331fb031b4e76b665f9a5272ce414211a986604"} Jul 02 23:20:24.593178 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:24.593133 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" podStartSLOduration=1.65331788 podStartE2EDuration="2.593118321s" totalImagesPullingTime="939.800441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:20:24.592039738 +0000 UTC m=+616.587288033" watchObservedRunningTime="2026-07-02 23:20:24.593118321 +0000 UTC m=+616.588366612" Jul 02 23:20:28.597231 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:28.597199 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:20:29.080018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.079983 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn"] Jul 02 23:20:29.083124 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.083101 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.085928 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.085906 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 02 23:20:29.094403 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.094380 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn"] Jul 02 23:20:29.147411 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.147375 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.147576 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.147420 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.147576 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.147487 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c078494b-4b9b-447b-8811-5fffd060aa27-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.147576 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.147526 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.147576 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.147554 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.147712 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.147624 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjjlc\" (UniqueName: \"kubernetes.io/projected/c078494b-4b9b-447b-8811-5fffd060aa27-kube-api-access-hjjlc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.248538 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.248495 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjjlc\" (UniqueName: \"kubernetes.io/projected/c078494b-4b9b-447b-8811-5fffd060aa27-kube-api-access-hjjlc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.248705 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.248556 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.248705 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.248591 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.248705 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.248629 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c078494b-4b9b-447b-8811-5fffd060aa27-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.248705 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.248653 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.248705 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.248681 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.249107 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.249073 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.249240 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.249087 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.249240 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.249143 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.250791 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.250770 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c078494b-4b9b-447b-8811-5fffd060aa27-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.251083 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.251063 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c078494b-4b9b-447b-8811-5fffd060aa27-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.256527 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.256508 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjjlc\" (UniqueName: \"kubernetes.io/projected/c078494b-4b9b-447b-8811-5fffd060aa27-kube-api-access-hjjlc\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn\" (UID: \"c078494b-4b9b-447b-8811-5fffd060aa27\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.393104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.393015 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:29.529522 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.529493 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn"] Jul 02 23:20:29.531358 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:20:29.531328 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc078494b_4b9b_447b_8811_5fffd060aa27.slice/crio-62b5f5504894458c183c68de978884fd8e3d83abe6e9bb3db6daec2271e3c0c5 WatchSource:0}: Error finding container 62b5f5504894458c183c68de978884fd8e3d83abe6e9bb3db6daec2271e3c0c5: Status 404 returned error can't find the container with id 62b5f5504894458c183c68de978884fd8e3d83abe6e9bb3db6daec2271e3c0c5 Jul 02 23:20:29.600241 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:29.600209 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" event={"ID":"c078494b-4b9b-447b-8811-5fffd060aa27","Type":"ContainerStarted","Data":"62b5f5504894458c183c68de978884fd8e3d83abe6e9bb3db6daec2271e3c0c5"} Jul 02 23:20:30.606475 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:30.606439 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" event={"ID":"c078494b-4b9b-447b-8811-5fffd060aa27","Type":"ContainerStarted","Data":"6e2ab56d3fb2fad308ca98eeeafe510f0fc11acc04d7ac24b47c5e34187e795c"} Jul 02 23:20:30.624177 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:30.624118 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" podStartSLOduration=1.461441332 podStartE2EDuration="1.62410346s" totalImagesPullingTime="162.662128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:20:30.622587924 +0000 UTC m=+622.617836214" watchObservedRunningTime="2026-07-02 23:20:30.62410346 +0000 UTC m=+622.619351751" Jul 02 23:20:32.584148 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:32.584100 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:32.584570 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:32.584194 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:32.596675 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:32.596650 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:32.628081 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:32.628056 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-xs4hr" Jul 02 23:20:35.401260 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:35.401222 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:20:39.393737 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:39.393700 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:39.394121 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:39.393751 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:39.406412 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:39.406386 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:20:39.670786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:20:39.670704 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn" Jul 02 23:22:18.810006 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:22:18.809972 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:22:29.699436 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:22:29.699392 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:22:40.702390 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:22:40.702354 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:22:48.806307 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:22:48.806275 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:22:59.696056 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:22:59.696021 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:24:05.598259 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:24:05.598223 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:24:14.208127 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:24:14.208079 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:24:24.098533 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:24:24.098493 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:24:29.896813 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:24:29.896778 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:24:40.401203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:24:40.401152 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:24:49.316774 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:24:49.316738 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:25:01.614307 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:25:01.614229 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:25:09.066382 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:25:09.066264 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:25:09.072107 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:25:09.069062 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:25:09.073542 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:25:09.073519 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:25:09.076110 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:25:09.076084 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:25:10.997889 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:25:10.997852 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:25:33.702272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:25:33.702234 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:25:45.998333 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:25:45.998299 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:26:08.001632 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:26:08.001598 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:26:19.505352 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:26:19.505314 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:26:34.097765 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:26:34.097686 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:26:45.299701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:26:45.299662 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:26:56.294659 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:26:56.294616 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:27:04.998264 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:27:04.998224 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:27:14.198344 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:27:14.198306 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:29:00.998629 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:29:00.998595 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:29:09.394449 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:29:09.394415 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:29:21.396236 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:29:21.396198 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:29:31.309519 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:29:31.309432 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:29:42.100576 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:29:42.100541 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:29:53.097992 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:29:53.097949 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:30:01.700031 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:01.699993 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:30:09.098143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:09.098107 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:30:09.102701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:09.102676 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:30:09.105010 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:09.104986 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:30:09.109749 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:09.109730 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:30:12.997566 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:12.997528 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:30:22.998201 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:22.998153 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:30:32.795117 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:32.795073 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:30:43.395648 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:30:43.395606 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:31:06.001037 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:31:06.001000 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:31:14.598370 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:31:14.598335 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:31:25.794258 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:31:25.794220 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:31:37.494716 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:31:37.494679 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:31:45.800215 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:31:45.800166 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:31:53.499779 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:31:53.499736 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:32:02.190831 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:32:02.190792 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:32:10.599870 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:32:10.599833 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:32:20.395477 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:32:20.395440 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:32:29.101501 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:32:29.101463 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:32:37.900896 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:32:37.900810 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:32:52.502668 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:32:52.502633 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:33:01.297427 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:33:01.297390 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:33:10.492831 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:33:10.492783 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:33:21.895104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:33:21.895066 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:33:30.695528 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:33:30.695483 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:33:36.993907 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:33:36.993869 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:33:53.492704 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:33:53.492658 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:34:04.398042 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:34:04.397997 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:34:13.197784 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:34:13.197744 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:34:19.997149 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:34:19.997109 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:34:32.096857 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:34:32.096811 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:34:40.498562 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:34:40.498527 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:34:49.803581 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:34:49.803534 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:35:06.106884 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:35:06.106840 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:35:09.131572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:35:09.131459 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:35:09.137441 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:35:09.137418 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:35:09.138310 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:35:09.138289 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:35:09.144285 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:35:09.144261 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:35:15.004470 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:35:15.004430 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:35:37.695417 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:35:37.695376 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:35:46.196517 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:35:46.196479 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:37:07.802570 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:37:07.802535 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:37:15.120097 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:37:15.115887 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:37:23.498880 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:37:23.498843 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:37:32.709155 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:37:32.709115 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:37:39.106662 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:37:39.106624 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:38:09.705878 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:38:09.705835 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:40:09.163486 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:40:09.163377 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:40:09.176492 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:40:09.176459 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:40:09.176632 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:40:09.176497 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:40:09.183343 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:40:09.183322 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:41:16.296849 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:41:16.296811 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:41:25.391337 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:41:25.391300 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:41:37.800460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:41:37.800379 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:42:24.609409 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:24.609374 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:42:34.514745 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:34.514704 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:42:55.492564 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.492529 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45"] Jul 02 23:42:55.495830 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.495806 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.499860 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.499834 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1ef970\"/\"test-model-1ef970-kserve-self-signed-certs\"" Jul 02 23:42:55.499997 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.499871 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1ef970\"/\"kube-root-ca.crt\"" Jul 02 23:42:55.499997 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.499838 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-1ef970\"/\"openshift-service-ca.crt\"" Jul 02 23:42:55.499997 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.499844 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-1ef970\"/\"default-dockercfg-rl4d4\"" Jul 02 23:42:55.508522 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.508493 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45"] Jul 02 23:42:55.624339 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.624295 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-home\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.624339 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.624348 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-dshm\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.624610 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.624370 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tmp-dir\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.624610 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.624397 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tls-certs\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.624610 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.624421 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-model-cache\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.624610 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.624555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k6kb\" (UniqueName: \"kubernetes.io/projected/a999f5d9-d196-4ed2-aa7b-c896b3363edf-kube-api-access-8k6kb\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.725997 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.725958 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-home\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.726174 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.726006 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-dshm\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.726174 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.726028 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tmp-dir\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.726174 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.726056 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tls-certs\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.726174 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.726084 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-model-cache\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.726350 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.726196 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k6kb\" (UniqueName: \"kubernetes.io/projected/a999f5d9-d196-4ed2-aa7b-c896b3363edf-kube-api-access-8k6kb\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.726465 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.726438 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-home\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.726510 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.726474 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tmp-dir\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.726584 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.726556 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-model-cache\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.728460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.728433 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-dshm\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.728693 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.728675 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tls-certs\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.737574 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.737546 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k6kb\" (UniqueName: \"kubernetes.io/projected/a999f5d9-d196-4ed2-aa7b-c896b3363edf-kube-api-access-8k6kb\") pod \"test-model-1ef970-kserve-58dc45bb4f-rwv45\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.806890 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.806781 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:42:55.943629 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:55.943603 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45"] Jul 02 23:42:55.945877 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:42:55.945849 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda999f5d9_d196_4ed2_aa7b_c896b3363edf.slice/crio-ae0acb79544ea644d31f366f5836905e183d041ee75696f8a9a1932ced336106 WatchSource:0}: Error finding container ae0acb79544ea644d31f366f5836905e183d041ee75696f8a9a1932ced336106: Status 404 returned error can't find the container with id ae0acb79544ea644d31f366f5836905e183d041ee75696f8a9a1932ced336106 Jul 02 23:42:56.494079 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:56.494035 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" event={"ID":"a999f5d9-d196-4ed2-aa7b-c896b3363edf","Type":"ContainerStarted","Data":"60d0f76d89b86e49c8696cc8da03332251b11991a752f2c39a26d55f03d48967"} Jul 02 23:42:56.494079 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:56.494084 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" event={"ID":"a999f5d9-d196-4ed2-aa7b-c896b3363edf","Type":"ContainerStarted","Data":"ae0acb79544ea644d31f366f5836905e183d041ee75696f8a9a1932ced336106"} Jul 02 23:42:56.511089 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:56.511033 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" podStartSLOduration=1.5110178730000001 podStartE2EDuration="1.511017873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:42:56.50989771 +0000 UTC m=+1968.505146020" watchObservedRunningTime="2026-07-02 23:42:56.511017873 +0000 UTC m=+1968.506266164" Jul 02 23:42:58.493302 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:42:58.493263 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:43:05.807175 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:05.807132 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:43:05.807175 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:05.807179 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:43:05.819622 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:05.819586 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:43:06.556840 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:06.556804 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:43:17.789988 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.789953 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z"] Jul 02 23:43:17.793072 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.793053 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:17.795846 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.795829 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-20977a\"/\"default-dockercfg-h6vfh\"" Jul 02 23:43:17.795960 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.795847 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-20977a\"/\"openshift-service-ca.crt\"" Jul 02 23:43:17.796917 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.796901 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-20977a\"/\"kube-root-ca.crt\"" Jul 02 23:43:17.796978 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.796964 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-20977a\"/\"test-model-20977a-kserve-self-signed-certs\"" Jul 02 23:43:17.801986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.801960 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z"] Jul 02 23:43:17.926328 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.926279 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-tmp-dir\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:17.926328 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.926332 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/129f9248-2337-40cb-b45c-fa146c8a9f70-tls-certs\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:17.926620 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.926353 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-home\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:17.926620 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.926460 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-model-cache\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:17.926620 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.926520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-dshm\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:17.926620 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:17.926564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz6vg\" (UniqueName: \"kubernetes.io/projected/129f9248-2337-40cb-b45c-fa146c8a9f70-kube-api-access-mz6vg\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.027450 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.027417 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz6vg\" (UniqueName: \"kubernetes.io/projected/129f9248-2337-40cb-b45c-fa146c8a9f70-kube-api-access-mz6vg\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.027636 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.027468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-tmp-dir\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.027636 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.027487 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/129f9248-2337-40cb-b45c-fa146c8a9f70-tls-certs\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.027636 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.027504 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-home\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.027636 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.027560 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-model-cache\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.027636 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.027605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-dshm\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.027904 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.027854 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-tmp-dir\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.027966 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.027922 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-home\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.028045 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.028025 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-model-cache\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.030037 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.030014 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-dshm\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.030275 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.030260 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/129f9248-2337-40cb-b45c-fa146c8a9f70-tls-certs\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.035786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.035767 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz6vg\" (UniqueName: \"kubernetes.io/projected/129f9248-2337-40cb-b45c-fa146c8a9f70-kube-api-access-mz6vg\") pod \"test-model-20977a-kserve-5f9f556677-6z77z\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.103835 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.103800 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:18.239411 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.239386 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z"] Jul 02 23:43:18.241086 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:43:18.241061 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod129f9248_2337_40cb_b45c_fa146c8a9f70.slice/crio-e755c17bbf7b3e7124c77369a5470a4cc46e40a1f7ba0114cf068605f32b27a7 WatchSource:0}: Error finding container e755c17bbf7b3e7124c77369a5470a4cc46e40a1f7ba0114cf068605f32b27a7: Status 404 returned error can't find the container with id e755c17bbf7b3e7124c77369a5470a4cc46e40a1f7ba0114cf068605f32b27a7 Jul 02 23:43:18.605159 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.605120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" event={"ID":"129f9248-2337-40cb-b45c-fa146c8a9f70","Type":"ContainerStarted","Data":"e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead"} Jul 02 23:43:18.605159 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.605165 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" event={"ID":"129f9248-2337-40cb-b45c-fa146c8a9f70","Type":"ContainerStarted","Data":"e755c17bbf7b3e7124c77369a5470a4cc46e40a1f7ba0114cf068605f32b27a7"} Jul 02 23:43:18.622888 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:18.622838 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" podStartSLOduration=1.622824267 podStartE2EDuration="1.622824267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:43:18.622512838 +0000 UTC m=+1990.617761132" watchObservedRunningTime="2026-07-02 23:43:18.622824267 +0000 UTC m=+1990.618072557" Jul 02 23:43:21.702226 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:21.702172 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:43:28.104940 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:28.104900 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:28.105530 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:28.104969 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:28.117517 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:28.117488 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:28.664778 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:28.664739 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:33.703131 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:33.703096 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:43:38.596782 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:38.596750 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45"] Jul 02 23:43:39.477629 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:39.477591 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z"] Jul 02 23:43:45.593806 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:45.593764 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:43:53.617845 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.617802 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" podUID="a999f5d9-d196-4ed2-aa7b-c896b3363edf" containerName="main" containerID="cri-o://60d0f76d89b86e49c8696cc8da03332251b11991a752f2c39a26d55f03d48967" gracePeriod=45 Jul 02 23:43:53.792901 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.792870 2570 generic.go:356] "Generic (PLEG): container finished" podID="a999f5d9-d196-4ed2-aa7b-c896b3363edf" containerID="60d0f76d89b86e49c8696cc8da03332251b11991a752f2c39a26d55f03d48967" exitCode=0 Jul 02 23:43:53.793048 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.792943 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" event={"ID":"a999f5d9-d196-4ed2-aa7b-c896b3363edf","Type":"ContainerDied","Data":"60d0f76d89b86e49c8696cc8da03332251b11991a752f2c39a26d55f03d48967"} Jul 02 23:43:53.877757 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.877691 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:43:53.966285 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966249 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-home\") pod \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " Jul 02 23:43:53.966480 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966304 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-model-cache\") pod \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " Jul 02 23:43:53.966480 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966352 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tls-certs\") pod \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " Jul 02 23:43:53.966480 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966368 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k6kb\" (UniqueName: \"kubernetes.io/projected/a999f5d9-d196-4ed2-aa7b-c896b3363edf-kube-api-access-8k6kb\") pod \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " Jul 02 23:43:53.966480 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966409 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-dshm\") pod \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " Jul 02 23:43:53.966480 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966425 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tmp-dir\") pod \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\" (UID: \"a999f5d9-d196-4ed2-aa7b-c896b3363edf\") " Jul 02 23:43:53.966756 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966574 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-home" (OuterVolumeSpecName: "home") pod "a999f5d9-d196-4ed2-aa7b-c896b3363edf" (UID: "a999f5d9-d196-4ed2-aa7b-c896b3363edf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:43:53.966756 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966623 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-model-cache" (OuterVolumeSpecName: "model-cache") pod "a999f5d9-d196-4ed2-aa7b-c896b3363edf" (UID: "a999f5d9-d196-4ed2-aa7b-c896b3363edf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:43:53.966756 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966717 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:53.966756 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966742 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:53.966756 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.966713 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a999f5d9-d196-4ed2-aa7b-c896b3363edf" (UID: "a999f5d9-d196-4ed2-aa7b-c896b3363edf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:43:53.968697 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.968668 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a999f5d9-d196-4ed2-aa7b-c896b3363edf" (UID: "a999f5d9-d196-4ed2-aa7b-c896b3363edf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:43:53.968814 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.968716 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a999f5d9-d196-4ed2-aa7b-c896b3363edf-kube-api-access-8k6kb" (OuterVolumeSpecName: "kube-api-access-8k6kb") pod "a999f5d9-d196-4ed2-aa7b-c896b3363edf" (UID: "a999f5d9-d196-4ed2-aa7b-c896b3363edf"). InnerVolumeSpecName "kube-api-access-8k6kb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:43:53.968814 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:53.968734 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-dshm" (OuterVolumeSpecName: "dshm") pod "a999f5d9-d196-4ed2-aa7b-c896b3363edf" (UID: "a999f5d9-d196-4ed2-aa7b-c896b3363edf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:43:54.067431 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.067392 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.067431 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.067425 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k6kb\" (UniqueName: \"kubernetes.io/projected/a999f5d9-d196-4ed2-aa7b-c896b3363edf-kube-api-access-8k6kb\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.067431 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.067438 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.067666 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.067450 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a999f5d9-d196-4ed2-aa7b-c896b3363edf-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.497857 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.497797 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" podUID="129f9248-2337-40cb-b45c-fa146c8a9f70" containerName="main" containerID="cri-o://e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead" gracePeriod=45 Jul 02 23:43:54.754800 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.754738 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:54.799547 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.799512 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" Jul 02 23:43:54.799547 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.799533 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45" event={"ID":"a999f5d9-d196-4ed2-aa7b-c896b3363edf","Type":"ContainerDied","Data":"ae0acb79544ea644d31f366f5836905e183d041ee75696f8a9a1932ced336106"} Jul 02 23:43:54.799781 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.799579 2570 scope.go:117] "RemoveContainer" containerID="60d0f76d89b86e49c8696cc8da03332251b11991a752f2c39a26d55f03d48967" Jul 02 23:43:54.800936 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.800910 2570 generic.go:356] "Generic (PLEG): container finished" podID="129f9248-2337-40cb-b45c-fa146c8a9f70" containerID="e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead" exitCode=0 Jul 02 23:43:54.801045 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.800946 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" Jul 02 23:43:54.801045 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.800994 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" event={"ID":"129f9248-2337-40cb-b45c-fa146c8a9f70","Type":"ContainerDied","Data":"e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead"} Jul 02 23:43:54.801045 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.801029 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z" event={"ID":"129f9248-2337-40cb-b45c-fa146c8a9f70","Type":"ContainerDied","Data":"e755c17bbf7b3e7124c77369a5470a4cc46e40a1f7ba0114cf068605f32b27a7"} Jul 02 23:43:54.808427 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.808407 2570 scope.go:117] "RemoveContainer" containerID="e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead" Jul 02 23:43:54.816505 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.816484 2570 scope.go:117] "RemoveContainer" containerID="e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead" Jul 02 23:43:54.816797 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:43:54.816777 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead\": container with ID starting with e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead not found: ID does not exist" containerID="e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead" Jul 02 23:43:54.816865 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.816804 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead"} err="failed to get container status \"e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead\": rpc error: code = NotFound desc = could not find container \"e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead\": container with ID starting with e0cba9bdce2d62afd9cf50f0caa35c7e9254a9fdcd18c99e644a5af4d5390ead not found: ID does not exist" Jul 02 23:43:54.821905 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.821862 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45"] Jul 02 23:43:54.823937 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.823911 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-1ef970/test-model-1ef970-kserve-58dc45bb4f-rwv45"] Jul 02 23:43:54.874529 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874499 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-tmp-dir\") pod \"129f9248-2337-40cb-b45c-fa146c8a9f70\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " Jul 02 23:43:54.874678 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874539 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-model-cache\") pod \"129f9248-2337-40cb-b45c-fa146c8a9f70\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " Jul 02 23:43:54.874678 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874595 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/129f9248-2337-40cb-b45c-fa146c8a9f70-tls-certs\") pod \"129f9248-2337-40cb-b45c-fa146c8a9f70\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " Jul 02 23:43:54.874678 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874614 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz6vg\" (UniqueName: \"kubernetes.io/projected/129f9248-2337-40cb-b45c-fa146c8a9f70-kube-api-access-mz6vg\") pod \"129f9248-2337-40cb-b45c-fa146c8a9f70\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " Jul 02 23:43:54.874678 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874641 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-home\") pod \"129f9248-2337-40cb-b45c-fa146c8a9f70\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " Jul 02 23:43:54.874678 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874668 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-dshm\") pod \"129f9248-2337-40cb-b45c-fa146c8a9f70\" (UID: \"129f9248-2337-40cb-b45c-fa146c8a9f70\") " Jul 02 23:43:54.874906 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874834 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "129f9248-2337-40cb-b45c-fa146c8a9f70" (UID: "129f9248-2337-40cb-b45c-fa146c8a9f70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:43:54.874906 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874858 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-model-cache" (OuterVolumeSpecName: "model-cache") pod "129f9248-2337-40cb-b45c-fa146c8a9f70" (UID: "129f9248-2337-40cb-b45c-fa146c8a9f70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:43:54.875015 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874932 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-home" (OuterVolumeSpecName: "home") pod "129f9248-2337-40cb-b45c-fa146c8a9f70" (UID: "129f9248-2337-40cb-b45c-fa146c8a9f70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:43:54.875015 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.874985 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.875015 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.875000 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.877012 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.876989 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-dshm" (OuterVolumeSpecName: "dshm") pod "129f9248-2337-40cb-b45c-fa146c8a9f70" (UID: "129f9248-2337-40cb-b45c-fa146c8a9f70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:43:54.877012 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.876999 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/129f9248-2337-40cb-b45c-fa146c8a9f70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "129f9248-2337-40cb-b45c-fa146c8a9f70" (UID: "129f9248-2337-40cb-b45c-fa146c8a9f70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:43:54.877126 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.877013 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/129f9248-2337-40cb-b45c-fa146c8a9f70-kube-api-access-mz6vg" (OuterVolumeSpecName: "kube-api-access-mz6vg") pod "129f9248-2337-40cb-b45c-fa146c8a9f70" (UID: "129f9248-2337-40cb-b45c-fa146c8a9f70"). InnerVolumeSpecName "kube-api-access-mz6vg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:43:54.975786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.975747 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/129f9248-2337-40cb-b45c-fa146c8a9f70-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.975786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.975778 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz6vg\" (UniqueName: \"kubernetes.io/projected/129f9248-2337-40cb-b45c-fa146c8a9f70-kube-api-access-mz6vg\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.975786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.975787 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:54.975786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:54.975795 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/129f9248-2337-40cb-b45c-fa146c8a9f70-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:43:55.125682 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:55.125643 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z"] Jul 02 23:43:55.130695 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:55.130658 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-20977a/test-model-20977a-kserve-5f9f556677-6z77z"] Jul 02 23:43:56.592534 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:56.592498 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="129f9248-2337-40cb-b45c-fa146c8a9f70" path="/var/lib/kubelet/pods/129f9248-2337-40cb-b45c-fa146c8a9f70/volumes" Jul 02 23:43:56.592920 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:43:56.592884 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a999f5d9-d196-4ed2-aa7b-c896b3363edf" path="/var/lib/kubelet/pods/a999f5d9-d196-4ed2-aa7b-c896b3363edf/volumes" Jul 02 23:44:26.803106 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:44:26.803066 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:44:39.308733 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:44:39.308634 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:44:49.398242 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:44:49.398198 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:45:00.152085 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.152043 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717265-mpz6k"] Jul 02 23:45:00.152515 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.152406 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a999f5d9-d196-4ed2-aa7b-c896b3363edf" containerName="main" Jul 02 23:45:00.152515 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.152418 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a999f5d9-d196-4ed2-aa7b-c896b3363edf" containerName="main" Jul 02 23:45:00.152515 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.152438 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129f9248-2337-40cb-b45c-fa146c8a9f70" containerName="main" Jul 02 23:45:00.152515 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.152443 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="129f9248-2337-40cb-b45c-fa146c8a9f70" containerName="main" Jul 02 23:45:00.152515 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.152499 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="129f9248-2337-40cb-b45c-fa146c8a9f70" containerName="main" Jul 02 23:45:00.152515 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.152509 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a999f5d9-d196-4ed2-aa7b-c896b3363edf" containerName="main" Jul 02 23:45:00.155506 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.155489 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" Jul 02 23:45:00.158752 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.158731 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-nmtfb\"" Jul 02 23:45:00.167281 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.167240 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717265-mpz6k"] Jul 02 23:45:00.230367 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.230322 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq6xv\" (UniqueName: \"kubernetes.io/projected/5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d-kube-api-access-rq6xv\") pod \"maas-api-key-cleanup-29717265-mpz6k\" (UID: \"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d\") " pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" Jul 02 23:45:00.330978 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.330933 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq6xv\" (UniqueName: \"kubernetes.io/projected/5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d-kube-api-access-rq6xv\") pod \"maas-api-key-cleanup-29717265-mpz6k\" (UID: \"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d\") " pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" Jul 02 23:45:00.339655 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.339627 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq6xv\" (UniqueName: \"kubernetes.io/projected/5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d-kube-api-access-rq6xv\") pod \"maas-api-key-cleanup-29717265-mpz6k\" (UID: \"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d\") " pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" Jul 02 23:45:00.467257 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.467161 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" Jul 02 23:45:00.606600 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.606574 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717265-mpz6k"] Jul 02 23:45:00.608725 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:45:00.608681 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfc161d_550e_4d7d_8264_0ecb4fe5fb7d.slice/crio-3b5586b665484a86603ca3a652d53fceb5e3f5d788de9b721c362b41514a3d85 WatchSource:0}: Error finding container 3b5586b665484a86603ca3a652d53fceb5e3f5d788de9b721c362b41514a3d85: Status 404 returned error can't find the container with id 3b5586b665484a86603ca3a652d53fceb5e3f5d788de9b721c362b41514a3d85 Jul 02 23:45:00.610621 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:00.610602 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:45:01.000885 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:01.000846 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:45:01.145107 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:01.145069 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" event={"ID":"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d","Type":"ContainerStarted","Data":"3b5586b665484a86603ca3a652d53fceb5e3f5d788de9b721c362b41514a3d85"} Jul 02 23:45:03.157277 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:03.157233 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" event={"ID":"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d","Type":"ContainerStarted","Data":"ddf20f69dcf08f3a83c9108e7247c557b69153455fc5c5641389dd3c7ffd6146"} Jul 02 23:45:03.177623 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:03.177571 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" podStartSLOduration=0.989773884 podStartE2EDuration="3.177552545s" totalImagesPullingTime="2.187778661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:45:03.175882542 +0000 UTC m=+2095.171130887" watchObservedRunningTime="2026-07-02 23:45:03.177552545 +0000 UTC m=+2095.172800839" Jul 02 23:45:04.164273 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:04.164234 2570 generic.go:356] "Generic (PLEG): container finished" podID="5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d" containerID="ddf20f69dcf08f3a83c9108e7247c557b69153455fc5c5641389dd3c7ffd6146" exitCode=0 Jul 02 23:45:04.164672 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:04.164316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" event={"ID":"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d","Type":"ContainerDied","Data":"ddf20f69dcf08f3a83c9108e7247c557b69153455fc5c5641389dd3c7ffd6146"} Jul 02 23:45:05.305625 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:05.305593 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" Jul 02 23:45:05.374965 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:05.374926 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq6xv\" (UniqueName: \"kubernetes.io/projected/5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d-kube-api-access-rq6xv\") pod \"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d\" (UID: \"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d\") " Jul 02 23:45:05.377231 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:05.377202 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d-kube-api-access-rq6xv" (OuterVolumeSpecName: "kube-api-access-rq6xv") pod "5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d" (UID: "5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d"). InnerVolumeSpecName "kube-api-access-rq6xv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:45:05.476250 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:05.476129 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq6xv\" (UniqueName: \"kubernetes.io/projected/5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d-kube-api-access-rq6xv\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:45:06.175572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:06.175539 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" event={"ID":"5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d","Type":"ContainerDied","Data":"3b5586b665484a86603ca3a652d53fceb5e3f5d788de9b721c362b41514a3d85"} Jul 02 23:45:06.175572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:06.175560 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717265-mpz6k" Jul 02 23:45:06.175786 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:06.175593 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b5586b665484a86603ca3a652d53fceb5e3f5d788de9b721c362b41514a3d85" Jul 02 23:45:09.206030 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:09.205919 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:45:09.218529 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:09.212900 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:45:09.218529 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:09.213156 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:45:09.220989 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:09.220966 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:45:10.369045 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:10.368994 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:45:20.707922 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:20.707887 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:45:31.303055 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:31.303020 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:45:34.073390 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.073354 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d"] Jul 02 23:45:34.073919 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.073900 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d" containerName="cleanup" Jul 02 23:45:34.074015 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.073924 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d" containerName="cleanup" Jul 02 23:45:34.074051 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.074024 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d" containerName="cleanup" Jul 02 23:45:34.080875 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.080853 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.084561 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.084522 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6d35bd\"/\"openshift-service-ca.crt\"" Jul 02 23:45:34.084561 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.084544 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6d35bd\"/\"default-dockercfg-5rlgh\"" Jul 02 23:45:34.084561 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.084551 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6d35bd\"/\"auth-test-model-6d35bd-kserve-self-signed-certs\"" Jul 02 23:45:34.084847 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.084550 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6d35bd\"/\"kube-root-ca.crt\"" Jul 02 23:45:34.085544 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.085518 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d"] Jul 02 23:45:34.232826 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.232789 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-home\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.232826 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.232827 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-dshm\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.233103 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.232868 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tmp-dir\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.233103 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.232941 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvrfp\" (UniqueName: \"kubernetes.io/projected/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-kube-api-access-pvrfp\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.233103 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.232987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tls-certs\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.233103 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.233014 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-model-cache\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.333751 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.333650 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-home\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.333751 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.333698 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-dshm\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.333987 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.333756 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tmp-dir\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.333987 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.333797 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvrfp\" (UniqueName: \"kubernetes.io/projected/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-kube-api-access-pvrfp\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.333987 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.333928 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tls-certs\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.333987 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.333970 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-model-cache\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.334223 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.334116 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-home\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.334273 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.334236 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tmp-dir\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.334379 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.334343 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-model-cache\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.335976 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.335953 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-dshm\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.336346 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.336329 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tls-certs\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.341346 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.341324 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvrfp\" (UniqueName: \"kubernetes.io/projected/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-kube-api-access-pvrfp\") pod \"auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.391698 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.391652 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:34.528161 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:34.528123 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d"] Jul 02 23:45:34.531879 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:45:34.531847 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c06e187_0b86_4b2b_8d46_691cb9ce2f54.slice/crio-e6a5a6b5da2a57be16091e4d363bac6e51247699cc1d6842f3c7a75b2e171861 WatchSource:0}: Error finding container e6a5a6b5da2a57be16091e4d363bac6e51247699cc1d6842f3c7a75b2e171861: Status 404 returned error can't find the container with id e6a5a6b5da2a57be16091e4d363bac6e51247699cc1d6842f3c7a75b2e171861 Jul 02 23:45:35.326648 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:35.326603 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" event={"ID":"5c06e187-0b86-4b2b-8d46-691cb9ce2f54","Type":"ContainerStarted","Data":"945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d"} Jul 02 23:45:35.327046 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:35.326657 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" event={"ID":"5c06e187-0b86-4b2b-8d46-691cb9ce2f54","Type":"ContainerStarted","Data":"e6a5a6b5da2a57be16091e4d363bac6e51247699cc1d6842f3c7a75b2e171861"} Jul 02 23:45:35.344491 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:35.344430 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" podStartSLOduration=1.344413448 podStartE2EDuration="1.344413448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:45:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:45:35.343038061 +0000 UTC m=+2127.338286372" watchObservedRunningTime="2026-07-02 23:45:35.344413448 +0000 UTC m=+2127.339661738" Jul 02 23:45:38.699836 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:38.699795 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:45:44.392202 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:44.392144 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:44.392695 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:44.392212 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:44.404779 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:44.404749 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:45:45.388875 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:45:45.388836 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:46:00.097453 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.097411 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t"] Jul 02 23:46:00.100759 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.100741 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.103519 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.103493 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"openshift-service-ca.crt\"" Jul 02 23:46:00.104783 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.104759 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"kube-root-ca.crt\"" Jul 02 23:46:00.104918 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.104890 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"auth-test-model-ed05a5-kserve-self-signed-certs\"" Jul 02 23:46:00.104918 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.104901 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"default-dockercfg-7p85f\"" Jul 02 23:46:00.111839 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.111817 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t"] Jul 02 23:46:00.169837 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.169805 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-home\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.170009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.169911 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tls-certs\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.170009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.169947 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-dshm\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.170009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.169968 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-model-cache\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.170009 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.169987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tmp-dir\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.170177 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.170068 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnc8\" (UniqueName: \"kubernetes.io/projected/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-kube-api-access-xqnc8\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271197 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271135 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-model-cache\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271411 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271243 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tmp-dir\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271411 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271294 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnc8\" (UniqueName: \"kubernetes.io/projected/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-kube-api-access-xqnc8\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271411 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271337 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-home\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271609 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271432 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tls-certs\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271609 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271458 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-dshm\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271719 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271653 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-model-cache\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271719 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271700 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tmp-dir\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.271806 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.271763 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-home\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.273757 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.273737 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-dshm\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.274045 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.274025 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tls-certs\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.279127 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.279097 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnc8\" (UniqueName: \"kubernetes.io/projected/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-kube-api-access-xqnc8\") pod \"auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.411939 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.411852 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:00.549329 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:00.549293 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t"] Jul 02 23:46:00.551257 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:46:00.551225 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddee5c81_a5c4_4b2d_b956_e9979d0d1f34.slice/crio-2987ff34a995ab6c57a53285f6ff045a597ec3906e368316821213df68883cbc WatchSource:0}: Error finding container 2987ff34a995ab6c57a53285f6ff045a597ec3906e368316821213df68883cbc: Status 404 returned error can't find the container with id 2987ff34a995ab6c57a53285f6ff045a597ec3906e368316821213df68883cbc Jul 02 23:46:01.471781 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:01.471735 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" event={"ID":"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34","Type":"ContainerStarted","Data":"52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c"} Jul 02 23:46:01.471781 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:01.471785 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" event={"ID":"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34","Type":"ContainerStarted","Data":"2987ff34a995ab6c57a53285f6ff045a597ec3906e368316821213df68883cbc"} Jul 02 23:46:01.489343 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:01.489289 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" podStartSLOduration=1.489272566 podStartE2EDuration="1.489272566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:46:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:46:01.488031004 +0000 UTC m=+2153.483279309" watchObservedRunningTime="2026-07-02 23:46:01.489272566 +0000 UTC m=+2153.484520857" Jul 02 23:46:03.397784 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:03.397748 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:46:10.412317 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:10.412276 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:10.412798 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:10.412332 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:10.424847 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:10.424815 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:10.528099 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:10.528071 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:46:21.103676 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:21.103641 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:46:34.894402 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:34.894366 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:46:50.695481 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:46:50.695437 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:47:04.600649 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:04.600617 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:47:21.992867 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:21.992828 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:47:37.199647 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:37.199565 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:47:49.097242 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:49.097205 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d"] Jul 02 23:47:49.992902 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:49.992870 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t"] Jul 02 23:47:53.597223 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:53.597175 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:47:57.096683 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.096636 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29"] Jul 02 23:47:57.100677 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.100650 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.103528 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.103498 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6d35bd\"/\"sub-test-model-6d35bd-kserve-self-signed-certs\"" Jul 02 23:47:57.110328 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.110293 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29"] Jul 02 23:47:57.222474 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.222440 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-tmp-dir\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.222671 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.222486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-home\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.222671 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.222518 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-dshm\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.222671 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.222561 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-model-cache\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.222846 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.222684 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5d552d-20ef-4569-b659-266fb59f3de0-tls-certs\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.222846 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.222715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dq8c\" (UniqueName: \"kubernetes.io/projected/3b5d552d-20ef-4569-b659-266fb59f3de0-kube-api-access-6dq8c\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324029 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.323984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-home\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324271 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.324048 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-dshm\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324271 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.324071 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-model-cache\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324271 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.324133 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5d552d-20ef-4569-b659-266fb59f3de0-tls-certs\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324271 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.324257 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dq8c\" (UniqueName: \"kubernetes.io/projected/3b5d552d-20ef-4569-b659-266fb59f3de0-kube-api-access-6dq8c\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324524 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.324324 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-tmp-dir\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324524 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.324480 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-home\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324524 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.324503 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-model-cache\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.324686 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.324664 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-tmp-dir\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.326945 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.326928 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-dshm\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.327345 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.327323 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5d552d-20ef-4569-b659-266fb59f3de0-tls-certs\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.333653 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.333629 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dq8c\" (UniqueName: \"kubernetes.io/projected/3b5d552d-20ef-4569-b659-266fb59f3de0-kube-api-access-6dq8c\") pod \"sub-test-model-6d35bd-kserve-54f7757475-hfb29\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.412644 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.412544 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:47:57.547468 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:57.547442 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29"] Jul 02 23:47:57.549087 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:47:57.549060 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b5d552d_20ef_4569_b659_266fb59f3de0.slice/crio-b482a62cd2f582a3e4c672c58e99563019ba4bee29f1dbcb054a19d39a71b028 WatchSource:0}: Error finding container b482a62cd2f582a3e4c672c58e99563019ba4bee29f1dbcb054a19d39a71b028: Status 404 returned error can't find the container with id b482a62cd2f582a3e4c672c58e99563019ba4bee29f1dbcb054a19d39a71b028 Jul 02 23:47:58.076664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:58.076624 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" event={"ID":"3b5d552d-20ef-4569-b659-266fb59f3de0","Type":"ContainerStarted","Data":"8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07"} Jul 02 23:47:58.076664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:58.076670 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" event={"ID":"3b5d552d-20ef-4569-b659-266fb59f3de0","Type":"ContainerStarted","Data":"b482a62cd2f582a3e4c672c58e99563019ba4bee29f1dbcb054a19d39a71b028"} Jul 02 23:47:58.094093 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:47:58.094045 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" podStartSLOduration=1.094030003 podStartE2EDuration="1.094030003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:47:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:47:58.092502835 +0000 UTC m=+2270.087751147" watchObservedRunningTime="2026-07-02 23:47:58.094030003 +0000 UTC m=+2270.089278294" Jul 02 23:48:04.117118 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.117077 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" podUID="5c06e187-0b86-4b2b-8d46-691cb9ce2f54" containerName="main" containerID="cri-o://945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d" gracePeriod=45 Jul 02 23:48:04.366727 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.366699 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:48:04.490614 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.490518 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-dshm\") pod \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " Jul 02 23:48:04.490614 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.490575 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvrfp\" (UniqueName: \"kubernetes.io/projected/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-kube-api-access-pvrfp\") pod \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " Jul 02 23:48:04.490857 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.490643 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tmp-dir\") pod \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " Jul 02 23:48:04.490857 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.490689 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tls-certs\") pod \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " Jul 02 23:48:04.490857 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.490726 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-model-cache\") pod \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " Jul 02 23:48:04.490857 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.490756 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-home\") pod \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\" (UID: \"5c06e187-0b86-4b2b-8d46-691cb9ce2f54\") " Jul 02 23:48:04.491093 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.490935 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c06e187-0b86-4b2b-8d46-691cb9ce2f54" (UID: "5c06e187-0b86-4b2b-8d46-691cb9ce2f54"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:04.491093 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.491013 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c06e187-0b86-4b2b-8d46-691cb9ce2f54" (UID: "5c06e187-0b86-4b2b-8d46-691cb9ce2f54"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:04.491247 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.491101 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-home" (OuterVolumeSpecName: "home") pod "5c06e187-0b86-4b2b-8d46-691cb9ce2f54" (UID: "5c06e187-0b86-4b2b-8d46-691cb9ce2f54"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:04.492902 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.492875 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-dshm" (OuterVolumeSpecName: "dshm") pod "5c06e187-0b86-4b2b-8d46-691cb9ce2f54" (UID: "5c06e187-0b86-4b2b-8d46-691cb9ce2f54"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:04.492902 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.492892 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-kube-api-access-pvrfp" (OuterVolumeSpecName: "kube-api-access-pvrfp") pod "5c06e187-0b86-4b2b-8d46-691cb9ce2f54" (UID: "5c06e187-0b86-4b2b-8d46-691cb9ce2f54"). InnerVolumeSpecName "kube-api-access-pvrfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:48:04.493047 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.492904 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c06e187-0b86-4b2b-8d46-691cb9ce2f54" (UID: "5c06e187-0b86-4b2b-8d46-691cb9ce2f54"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:48:04.592166 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.592132 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:04.592166 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.592160 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvrfp\" (UniqueName: \"kubernetes.io/projected/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-kube-api-access-pvrfp\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:04.592407 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.592175 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:04.592407 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.592204 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:04.592407 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.592215 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:04.592407 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:04.592231 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c06e187-0b86-4b2b-8d46-691cb9ce2f54-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:05.014741 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.014690 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" podUID="ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" containerName="main" containerID="cri-o://52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c" gracePeriod=45 Jul 02 23:48:05.113519 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.113486 2570 generic.go:356] "Generic (PLEG): container finished" podID="5c06e187-0b86-4b2b-8d46-691cb9ce2f54" containerID="945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d" exitCode=0 Jul 02 23:48:05.113702 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.113532 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" Jul 02 23:48:05.113702 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.113572 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" event={"ID":"5c06e187-0b86-4b2b-8d46-691cb9ce2f54","Type":"ContainerDied","Data":"945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d"} Jul 02 23:48:05.113702 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.113620 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d" event={"ID":"5c06e187-0b86-4b2b-8d46-691cb9ce2f54","Type":"ContainerDied","Data":"e6a5a6b5da2a57be16091e4d363bac6e51247699cc1d6842f3c7a75b2e171861"} Jul 02 23:48:05.113702 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.113638 2570 scope.go:117] "RemoveContainer" containerID="945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d" Jul 02 23:48:05.131955 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.131898 2570 scope.go:117] "RemoveContainer" containerID="945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d" Jul 02 23:48:05.132627 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:48:05.132601 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d\": container with ID starting with 945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d not found: ID does not exist" containerID="945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d" Jul 02 23:48:05.132725 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.132640 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d"} err="failed to get container status \"945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d\": rpc error: code = NotFound desc = could not find container \"945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d\": container with ID starting with 945ced5df5c573348d878805b5e2327fc135af3a7460b72170b8b8ad7de4ac3d not found: ID does not exist" Jul 02 23:48:05.134394 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.134370 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d"] Jul 02 23:48:05.137422 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.137400 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/auth-test-model-6d35bd-kserve-bcb4dfd55-sdk2d"] Jul 02 23:48:05.271642 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.271574 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:48:05.300398 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.300355 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-home\") pod \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " Jul 02 23:48:05.300601 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.300419 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqnc8\" (UniqueName: \"kubernetes.io/projected/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-kube-api-access-xqnc8\") pod \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " Jul 02 23:48:05.300601 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.300452 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tmp-dir\") pod \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " Jul 02 23:48:05.300601 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.300507 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tls-certs\") pod \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " Jul 02 23:48:05.300777 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.300610 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-dshm\") pod \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " Jul 02 23:48:05.300777 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.300661 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-model-cache\") pod \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\" (UID: \"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34\") " Jul 02 23:48:05.300777 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.300664 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-home" (OuterVolumeSpecName: "home") pod "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" (UID: "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:05.300944 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.300808 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" (UID: "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:05.301037 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.301015 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:05.301037 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.301024 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-model-cache" (OuterVolumeSpecName: "model-cache") pod "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" (UID: "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:05.301148 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.301037 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:05.302886 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.302851 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-kube-api-access-xqnc8" (OuterVolumeSpecName: "kube-api-access-xqnc8") pod "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" (UID: "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34"). InnerVolumeSpecName "kube-api-access-xqnc8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:48:05.303023 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.302893 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-dshm" (OuterVolumeSpecName: "dshm") pod "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" (UID: "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:48:05.303203 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.303160 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" (UID: "ddee5c81-a5c4-4b2d-b956-e9979d0d1f34"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:48:05.402064 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.402023 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:05.402064 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.402056 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqnc8\" (UniqueName: \"kubernetes.io/projected/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-kube-api-access-xqnc8\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:05.402064 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.402067 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:05.402345 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:05.402077 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:48:06.121010 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.120974 2570 generic.go:356] "Generic (PLEG): container finished" podID="ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" containerID="52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c" exitCode=0 Jul 02 23:48:06.121238 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.121018 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" Jul 02 23:48:06.121238 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.121035 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" event={"ID":"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34","Type":"ContainerDied","Data":"52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c"} Jul 02 23:48:06.121238 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.121072 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t" event={"ID":"ddee5c81-a5c4-4b2d-b956-e9979d0d1f34","Type":"ContainerDied","Data":"2987ff34a995ab6c57a53285f6ff045a597ec3906e368316821213df68883cbc"} Jul 02 23:48:06.121238 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.121093 2570 scope.go:117] "RemoveContainer" containerID="52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c" Jul 02 23:48:06.135480 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.135240 2570 scope.go:117] "RemoveContainer" containerID="52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c" Jul 02 23:48:06.135754 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:48:06.135735 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c\": container with ID starting with 52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c not found: ID does not exist" containerID="52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c" Jul 02 23:48:06.135796 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.135764 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c"} err="failed to get container status \"52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c\": rpc error: code = NotFound desc = could not find container \"52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c\": container with ID starting with 52bd3bc767fddf5678ae497e5f42440b7248e089313c785ef0f1b635dc22737c not found: ID does not exist" Jul 02 23:48:06.146232 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.146206 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t"] Jul 02 23:48:06.149598 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.149578 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/auth-test-model-ed05a5-kserve-698bdb7fd7-mvn2t"] Jul 02 23:48:06.592593 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.592556 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c06e187-0b86-4b2b-8d46-691cb9ce2f54" path="/var/lib/kubelet/pods/5c06e187-0b86-4b2b-8d46-691cb9ce2f54/volumes" Jul 02 23:48:06.592956 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:06.592942 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" path="/var/lib/kubelet/pods/ddee5c81-a5c4-4b2d-b956-e9979d0d1f34/volumes" Jul 02 23:48:07.301708 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:07.301672 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:48:07.413238 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:07.413202 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:48:07.413238 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:07.413248 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:48:07.425777 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:07.425738 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:48:08.143773 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:08.143744 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:48:15.987959 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.987915 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9"] Jul 02 23:48:15.988460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.988327 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c06e187-0b86-4b2b-8d46-691cb9ce2f54" containerName="main" Jul 02 23:48:15.988460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.988342 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c06e187-0b86-4b2b-8d46-691cb9ce2f54" containerName="main" Jul 02 23:48:15.988460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.988363 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" containerName="main" Jul 02 23:48:15.988460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.988368 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" containerName="main" Jul 02 23:48:15.988460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.988428 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddee5c81-a5c4-4b2d-b956-e9979d0d1f34" containerName="main" Jul 02 23:48:15.988460 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.988437 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c06e187-0b86-4b2b-8d46-691cb9ce2f54" containerName="main" Jul 02 23:48:15.991367 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.991351 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:15.994327 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.994305 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"kube-root-ca.crt\"" Jul 02 23:48:15.995822 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.995800 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"default-dockercfg-7p85f\"" Jul 02 23:48:15.995822 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.995814 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"sub-test-model-ed05a5-kserve-self-signed-certs\"" Jul 02 23:48:15.996016 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:15.996004 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"openshift-service-ca.crt\"" Jul 02 23:48:16.001316 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.001294 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9"] Jul 02 23:48:16.094534 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.094493 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-dshm\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.094534 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.094535 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rtp6\" (UniqueName: \"kubernetes.io/projected/5e105863-7521-47d5-81ad-0caa7073239f-kube-api-access-2rtp6\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.094805 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.094554 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-home\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.094805 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.094721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-model-cache\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.094805 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.094764 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e105863-7521-47d5-81ad-0caa7073239f-tls-certs\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.094933 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.094821 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-tmp-dir\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.195527 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.195489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-model-cache\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.195527 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.195533 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e105863-7521-47d5-81ad-0caa7073239f-tls-certs\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.195780 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.195557 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-tmp-dir\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.195780 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.195604 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-dshm\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.195780 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.195627 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rtp6\" (UniqueName: \"kubernetes.io/projected/5e105863-7521-47d5-81ad-0caa7073239f-kube-api-access-2rtp6\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.195780 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.195654 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-home\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.196002 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.195946 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-model-cache\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.196002 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.195957 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-tmp-dir\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.196100 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.196028 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-home\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.197964 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.197935 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-dshm\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.198258 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.198241 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e105863-7521-47d5-81ad-0caa7073239f-tls-certs\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.203880 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.203860 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rtp6\" (UniqueName: \"kubernetes.io/projected/5e105863-7521-47d5-81ad-0caa7073239f-kube-api-access-2rtp6\") pod \"sub-test-model-ed05a5-kserve-55d78db767-kt8w9\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.301350 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.301253 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:16.437370 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:16.437338 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9"] Jul 02 23:48:16.439083 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:48:16.439055 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e105863_7521_47d5_81ad_0caa7073239f.slice/crio-7bdd43648647d0770b8f98fb17e9543d989de4c5acc73328ba075c67b1b83e46 WatchSource:0}: Error finding container 7bdd43648647d0770b8f98fb17e9543d989de4c5acc73328ba075c67b1b83e46: Status 404 returned error can't find the container with id 7bdd43648647d0770b8f98fb17e9543d989de4c5acc73328ba075c67b1b83e46 Jul 02 23:48:17.183453 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:17.183411 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" event={"ID":"5e105863-7521-47d5-81ad-0caa7073239f","Type":"ContainerStarted","Data":"4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1"} Jul 02 23:48:17.183453 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:17.183457 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" event={"ID":"5e105863-7521-47d5-81ad-0caa7073239f","Type":"ContainerStarted","Data":"7bdd43648647d0770b8f98fb17e9543d989de4c5acc73328ba075c67b1b83e46"} Jul 02 23:48:17.203273 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:17.203208 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" podStartSLOduration=2.203192179 podStartE2EDuration="2.203192179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:48:17.200681837 +0000 UTC m=+2289.195930130" watchObservedRunningTime="2026-07-02 23:48:17.203192179 +0000 UTC m=+2289.198440461" Jul 02 23:48:21.198836 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:21.198804 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:48:26.301410 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:26.301371 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:26.301934 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:26.301441 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:26.314237 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:26.314177 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:27.247625 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:27.247592 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:48:39.398866 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:39.398829 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:48:56.393767 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:48:56.393731 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:49:01.772925 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:01.772835 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29"] Jul 02 23:49:02.694497 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:02.694455 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9"] Jul 02 23:49:08.581300 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.581267 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw"] Jul 02 23:49:08.585159 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.585139 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.587840 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.587816 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6d35bd\"/\"rate-test-model-6d35bd-kserve-self-signed-certs\"" Jul 02 23:49:08.595434 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.595408 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw"] Jul 02 23:49:08.643447 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.643411 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67w87\" (UniqueName: \"kubernetes.io/projected/8a1c4331-d2a9-45e9-8a94-a21684f3f986-kube-api-access-67w87\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.643447 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.643452 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-dshm\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.643697 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.643479 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tls-certs\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.643697 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.643566 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-model-cache\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.643697 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.643618 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tmp-dir\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.643697 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.643642 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-home\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.744368 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744325 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67w87\" (UniqueName: \"kubernetes.io/projected/8a1c4331-d2a9-45e9-8a94-a21684f3f986-kube-api-access-67w87\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.744368 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-dshm\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.744637 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744395 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tls-certs\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.744637 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744425 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-model-cache\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.744637 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744462 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tmp-dir\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.744637 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744490 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-home\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.744931 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744903 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-model-cache\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.745040 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744930 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-home\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.745040 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.744959 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tmp-dir\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.746781 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.746761 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-dshm\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.747043 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.747026 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tls-certs\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.753077 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.753053 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67w87\" (UniqueName: \"kubernetes.io/projected/8a1c4331-d2a9-45e9-8a94-a21684f3f986-kube-api-access-67w87\") pod \"rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:08.897836 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:08.897740 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:09.035064 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:09.035032 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw"] Jul 02 23:49:09.036641 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:49:09.036616 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a1c4331_d2a9_45e9_8a94_a21684f3f986.slice/crio-5c9ddb0d8f62e8510130c2804aebec6a1b2cdbf27af09713d28b907f23083f75 WatchSource:0}: Error finding container 5c9ddb0d8f62e8510130c2804aebec6a1b2cdbf27af09713d28b907f23083f75: Status 404 returned error can't find the container with id 5c9ddb0d8f62e8510130c2804aebec6a1b2cdbf27af09713d28b907f23083f75 Jul 02 23:49:09.463473 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:09.463432 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" event={"ID":"8a1c4331-d2a9-45e9-8a94-a21684f3f986","Type":"ContainerStarted","Data":"b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464"} Jul 02 23:49:09.463473 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:09.463476 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" event={"ID":"8a1c4331-d2a9-45e9-8a94-a21684f3f986","Type":"ContainerStarted","Data":"5c9ddb0d8f62e8510130c2804aebec6a1b2cdbf27af09713d28b907f23083f75"} Jul 02 23:49:09.483868 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:09.483805 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" podStartSLOduration=1.4837894010000001 podStartE2EDuration="1.483789401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:49:09.481536077 +0000 UTC m=+2341.476784395" watchObservedRunningTime="2026-07-02 23:49:09.483789401 +0000 UTC m=+2341.479037691" Jul 02 23:49:12.992077 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:12.992044 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:49:16.794018 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:16.793953 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" podUID="3b5d552d-20ef-4569-b659-266fb59f3de0" containerName="main" containerID="cri-o://8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07" gracePeriod=45 Jul 02 23:49:17.047841 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.047776 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:49:17.109448 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109415 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-dshm\") pod \"3b5d552d-20ef-4569-b659-266fb59f3de0\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " Jul 02 23:49:17.109448 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109454 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-tmp-dir\") pod \"3b5d552d-20ef-4569-b659-266fb59f3de0\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " Jul 02 23:49:17.109700 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109474 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5d552d-20ef-4569-b659-266fb59f3de0-tls-certs\") pod \"3b5d552d-20ef-4569-b659-266fb59f3de0\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " Jul 02 23:49:17.109700 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109518 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-model-cache\") pod \"3b5d552d-20ef-4569-b659-266fb59f3de0\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " Jul 02 23:49:17.109700 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109540 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-home\") pod \"3b5d552d-20ef-4569-b659-266fb59f3de0\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " Jul 02 23:49:17.109700 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109557 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dq8c\" (UniqueName: \"kubernetes.io/projected/3b5d552d-20ef-4569-b659-266fb59f3de0-kube-api-access-6dq8c\") pod \"3b5d552d-20ef-4569-b659-266fb59f3de0\" (UID: \"3b5d552d-20ef-4569-b659-266fb59f3de0\") " Jul 02 23:49:17.109915 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109753 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b5d552d-20ef-4569-b659-266fb59f3de0" (UID: "3b5d552d-20ef-4569-b659-266fb59f3de0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:17.109915 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109800 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b5d552d-20ef-4569-b659-266fb59f3de0" (UID: "3b5d552d-20ef-4569-b659-266fb59f3de0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:17.109915 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.109820 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-home" (OuterVolumeSpecName: "home") pod "3b5d552d-20ef-4569-b659-266fb59f3de0" (UID: "3b5d552d-20ef-4569-b659-266fb59f3de0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:17.111633 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.111607 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-dshm" (OuterVolumeSpecName: "dshm") pod "3b5d552d-20ef-4569-b659-266fb59f3de0" (UID: "3b5d552d-20ef-4569-b659-266fb59f3de0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:17.111739 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.111634 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b5d552d-20ef-4569-b659-266fb59f3de0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b5d552d-20ef-4569-b659-266fb59f3de0" (UID: "3b5d552d-20ef-4569-b659-266fb59f3de0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:49:17.111794 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.111730 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b5d552d-20ef-4569-b659-266fb59f3de0-kube-api-access-6dq8c" (OuterVolumeSpecName: "kube-api-access-6dq8c") pod "3b5d552d-20ef-4569-b659-266fb59f3de0" (UID: "3b5d552d-20ef-4569-b659-266fb59f3de0"). InnerVolumeSpecName "kube-api-access-6dq8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:49:17.210351 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.210305 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:17.210351 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.210340 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:17.210351 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.210351 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dq8c\" (UniqueName: \"kubernetes.io/projected/3b5d552d-20ef-4569-b659-266fb59f3de0-kube-api-access-6dq8c\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:17.210351 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.210361 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:17.210630 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.210370 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b5d552d-20ef-4569-b659-266fb59f3de0-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:17.210630 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.210378 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b5d552d-20ef-4569-b659-266fb59f3de0-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:17.506384 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.506345 2570 generic.go:356] "Generic (PLEG): container finished" podID="3b5d552d-20ef-4569-b659-266fb59f3de0" containerID="8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07" exitCode=0 Jul 02 23:49:17.506572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.506392 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" Jul 02 23:49:17.506572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.506414 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" event={"ID":"3b5d552d-20ef-4569-b659-266fb59f3de0","Type":"ContainerDied","Data":"8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07"} Jul 02 23:49:17.506572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.506447 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29" event={"ID":"3b5d552d-20ef-4569-b659-266fb59f3de0","Type":"ContainerDied","Data":"b482a62cd2f582a3e4c672c58e99563019ba4bee29f1dbcb054a19d39a71b028"} Jul 02 23:49:17.506572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.506463 2570 scope.go:117] "RemoveContainer" containerID="8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07" Jul 02 23:49:17.516803 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.516779 2570 scope.go:117] "RemoveContainer" containerID="8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07" Jul 02 23:49:17.517079 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:49:17.517059 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07\": container with ID starting with 8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07 not found: ID does not exist" containerID="8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07" Jul 02 23:49:17.517151 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.517088 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07"} err="failed to get container status \"8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07\": rpc error: code = NotFound desc = could not find container \"8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07\": container with ID starting with 8acac9290b8412676251be762d3f0d50f952a2acdc81ed141d467bd8e9883e07 not found: ID does not exist" Jul 02 23:49:17.530118 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.530089 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29"] Jul 02 23:49:17.533604 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.533574 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/sub-test-model-6d35bd-kserve-54f7757475-hfb29"] Jul 02 23:49:17.717957 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.717909 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" podUID="5e105863-7521-47d5-81ad-0caa7073239f" containerName="main" containerID="cri-o://4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1" gracePeriod=45 Jul 02 23:49:17.974127 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:17.974103 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:49:18.016626 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.016590 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-model-cache\") pod \"5e105863-7521-47d5-81ad-0caa7073239f\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " Jul 02 23:49:18.016626 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.016630 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e105863-7521-47d5-81ad-0caa7073239f-tls-certs\") pod \"5e105863-7521-47d5-81ad-0caa7073239f\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " Jul 02 23:49:18.016877 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.016686 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rtp6\" (UniqueName: \"kubernetes.io/projected/5e105863-7521-47d5-81ad-0caa7073239f-kube-api-access-2rtp6\") pod \"5e105863-7521-47d5-81ad-0caa7073239f\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " Jul 02 23:49:18.016877 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.016731 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-dshm\") pod \"5e105863-7521-47d5-81ad-0caa7073239f\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " Jul 02 23:49:18.016877 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.016759 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-home\") pod \"5e105863-7521-47d5-81ad-0caa7073239f\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " Jul 02 23:49:18.016877 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.016804 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-tmp-dir\") pod \"5e105863-7521-47d5-81ad-0caa7073239f\" (UID: \"5e105863-7521-47d5-81ad-0caa7073239f\") " Jul 02 23:49:18.017168 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.016947 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e105863-7521-47d5-81ad-0caa7073239f" (UID: "5e105863-7521-47d5-81ad-0caa7073239f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:18.017168 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.017078 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:18.017168 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.017148 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e105863-7521-47d5-81ad-0caa7073239f" (UID: "5e105863-7521-47d5-81ad-0caa7073239f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:18.017307 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.017170 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-home" (OuterVolumeSpecName: "home") pod "5e105863-7521-47d5-81ad-0caa7073239f" (UID: "5e105863-7521-47d5-81ad-0caa7073239f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:18.019057 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.019032 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e105863-7521-47d5-81ad-0caa7073239f-kube-api-access-2rtp6" (OuterVolumeSpecName: "kube-api-access-2rtp6") pod "5e105863-7521-47d5-81ad-0caa7073239f" (UID: "5e105863-7521-47d5-81ad-0caa7073239f"). InnerVolumeSpecName "kube-api-access-2rtp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:49:18.019359 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.019339 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-dshm" (OuterVolumeSpecName: "dshm") pod "5e105863-7521-47d5-81ad-0caa7073239f" (UID: "5e105863-7521-47d5-81ad-0caa7073239f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:49:18.019425 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.019353 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e105863-7521-47d5-81ad-0caa7073239f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e105863-7521-47d5-81ad-0caa7073239f" (UID: "5e105863-7521-47d5-81ad-0caa7073239f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:49:18.118131 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.118090 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rtp6\" (UniqueName: \"kubernetes.io/projected/5e105863-7521-47d5-81ad-0caa7073239f-kube-api-access-2rtp6\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:18.118131 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.118126 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:18.118131 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.118137 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:18.118398 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.118146 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e105863-7521-47d5-81ad-0caa7073239f-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:18.118398 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.118156 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e105863-7521-47d5-81ad-0caa7073239f-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:49:18.513461 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.513369 2570 generic.go:356] "Generic (PLEG): container finished" podID="5e105863-7521-47d5-81ad-0caa7073239f" containerID="4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1" exitCode=0 Jul 02 23:49:18.513461 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.513416 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" Jul 02 23:49:18.513461 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.513451 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" event={"ID":"5e105863-7521-47d5-81ad-0caa7073239f","Type":"ContainerDied","Data":"4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1"} Jul 02 23:49:18.513683 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.513494 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9" event={"ID":"5e105863-7521-47d5-81ad-0caa7073239f","Type":"ContainerDied","Data":"7bdd43648647d0770b8f98fb17e9543d989de4c5acc73328ba075c67b1b83e46"} Jul 02 23:49:18.513683 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.513511 2570 scope.go:117] "RemoveContainer" containerID="4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1" Jul 02 23:49:18.526296 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.526272 2570 scope.go:117] "RemoveContainer" containerID="4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1" Jul 02 23:49:18.526613 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:49:18.526582 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1\": container with ID starting with 4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1 not found: ID does not exist" containerID="4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1" Jul 02 23:49:18.526664 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.526623 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1"} err="failed to get container status \"4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1\": rpc error: code = NotFound desc = could not find container \"4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1\": container with ID starting with 4a28f9d3d7273a9cc78a9b56a1b6d5660f1ce493d0f6d113a41916342b90e2d1 not found: ID does not exist" Jul 02 23:49:18.537683 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.537654 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9"] Jul 02 23:49:18.543762 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.543730 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/sub-test-model-ed05a5-kserve-55d78db767-kt8w9"] Jul 02 23:49:18.592543 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.592514 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b5d552d-20ef-4569-b659-266fb59f3de0" path="/var/lib/kubelet/pods/3b5d552d-20ef-4569-b659-266fb59f3de0/volumes" Jul 02 23:49:18.592911 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.592898 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e105863-7521-47d5-81ad-0caa7073239f" path="/var/lib/kubelet/pods/5e105863-7521-47d5-81ad-0caa7073239f/volumes" Jul 02 23:49:18.898385 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.898338 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:18.898546 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.898401 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:18.910893 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:18.910870 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:19.530304 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:19.530274 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:49:37.087382 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.087347 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45"] Jul 02 23:49:37.087873 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.087787 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b5d552d-20ef-4569-b659-266fb59f3de0" containerName="main" Jul 02 23:49:37.087873 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.087801 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b5d552d-20ef-4569-b659-266fb59f3de0" containerName="main" Jul 02 23:49:37.087873 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.087816 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e105863-7521-47d5-81ad-0caa7073239f" containerName="main" Jul 02 23:49:37.087873 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.087821 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e105863-7521-47d5-81ad-0caa7073239f" containerName="main" Jul 02 23:49:37.088027 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.087885 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e105863-7521-47d5-81ad-0caa7073239f" containerName="main" Jul 02 23:49:37.088027 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.087896 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b5d552d-20ef-4569-b659-266fb59f3de0" containerName="main" Jul 02 23:49:37.092112 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.092095 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.096337 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.096312 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"rate-test-model-ed05a5-kserve-self-signed-certs\"" Jul 02 23:49:37.096337 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.096332 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"openshift-service-ca.crt\"" Jul 02 23:49:37.096554 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.096339 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"kube-root-ca.crt\"" Jul 02 23:49:37.096554 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.096446 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ed05a5\"/\"default-dockercfg-7p85f\"" Jul 02 23:49:37.101401 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.101376 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45"] Jul 02 23:49:37.188858 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.188819 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-home\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.189049 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.188866 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tls-certs\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.189049 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.188925 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-dshm\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.189049 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.188977 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn592\" (UniqueName: \"kubernetes.io/projected/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-kube-api-access-zn592\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.189049 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.189008 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-model-cache\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.189218 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.189066 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tmp-dir\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.290432 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290392 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn592\" (UniqueName: \"kubernetes.io/projected/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-kube-api-access-zn592\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.290572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290446 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-model-cache\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.290572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290517 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tmp-dir\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.290572 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-home\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.290701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290596 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tls-certs\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.290701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290619 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-dshm\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.290963 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290936 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-model-cache\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.291044 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290951 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tmp-dir\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.291044 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.290987 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-home\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.292891 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.292867 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-dshm\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.293171 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.293150 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tls-certs\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.298723 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.298691 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn592\" (UniqueName: \"kubernetes.io/projected/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-kube-api-access-zn592\") pod \"rate-test-model-ed05a5-kserve-67f964c967-njf45\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.402935 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.402813 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:37.540461 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.540429 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45"] Jul 02 23:49:37.542084 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:49:37.542051 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9cba7b3_dd57_41ce_a544_0d1327f3aca3.slice/crio-0610bfa9f7ec81840d73a453beb12059bf05ff9f28495ae24a62b2ff01704058 WatchSource:0}: Error finding container 0610bfa9f7ec81840d73a453beb12059bf05ff9f28495ae24a62b2ff01704058: Status 404 returned error can't find the container with id 0610bfa9f7ec81840d73a453beb12059bf05ff9f28495ae24a62b2ff01704058 Jul 02 23:49:37.613678 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:37.613646 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" event={"ID":"c9cba7b3-dd57-41ce-a544-0d1327f3aca3","Type":"ContainerStarted","Data":"0610bfa9f7ec81840d73a453beb12059bf05ff9f28495ae24a62b2ff01704058"} Jul 02 23:49:38.620564 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:38.620527 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" event={"ID":"c9cba7b3-dd57-41ce-a544-0d1327f3aca3","Type":"ContainerStarted","Data":"c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64"} Jul 02 23:49:41.396826 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:41.396764 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" podStartSLOduration=4.396746661 podStartE2EDuration="4.396746661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:49:38.638824327 +0000 UTC m=+2370.634072621" watchObservedRunningTime="2026-07-02 23:49:41.396746661 +0000 UTC m=+2373.391994973" Jul 02 23:49:41.398104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:41.398077 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:49:47.403245 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:47.403180 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:47.403651 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:47.403269 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:47.415743 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:47.415714 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:49:47.676821 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:49:47.676728 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:50:04.102330 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:04.102287 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:50:09.239798 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:09.239684 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:50:09.247016 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:09.246989 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:50:09.249271 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:09.249248 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:50:09.256038 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:09.256011 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:50:23.299003 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:23.298963 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:50:30.093713 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:30.093673 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw"] Jul 02 23:50:30.988720 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:30.988633 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45"] Jul 02 23:50:37.695496 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:37.695460 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:50:45.114162 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.114114 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" podUID="8a1c4331-d2a9-45e9-8a94-a21684f3f986" containerName="main" containerID="cri-o://b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464" gracePeriod=45 Jul 02 23:50:45.359195 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.359157 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:50:45.500883 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.500780 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-model-cache\") pod \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " Jul 02 23:50:45.500883 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.500827 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-home\") pod \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " Jul 02 23:50:45.501125 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.500891 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tls-certs\") pod \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " Jul 02 23:50:45.501125 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.500930 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tmp-dir\") pod \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " Jul 02 23:50:45.501125 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.500963 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67w87\" (UniqueName: \"kubernetes.io/projected/8a1c4331-d2a9-45e9-8a94-a21684f3f986-kube-api-access-67w87\") pod \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " Jul 02 23:50:45.501125 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.501050 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-dshm\") pod \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\" (UID: \"8a1c4331-d2a9-45e9-8a94-a21684f3f986\") " Jul 02 23:50:45.501125 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.501079 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a1c4331-d2a9-45e9-8a94-a21684f3f986" (UID: "8a1c4331-d2a9-45e9-8a94-a21684f3f986"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:45.501430 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.501151 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-home" (OuterVolumeSpecName: "home") pod "8a1c4331-d2a9-45e9-8a94-a21684f3f986" (UID: "8a1c4331-d2a9-45e9-8a94-a21684f3f986"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:45.501430 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.501270 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a1c4331-d2a9-45e9-8a94-a21684f3f986" (UID: "8a1c4331-d2a9-45e9-8a94-a21684f3f986"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:45.501430 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.501320 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:45.501430 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.501336 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:45.503173 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.503150 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a1c4331-d2a9-45e9-8a94-a21684f3f986" (UID: "8a1c4331-d2a9-45e9-8a94-a21684f3f986"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:50:45.503173 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.503160 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-dshm" (OuterVolumeSpecName: "dshm") pod "8a1c4331-d2a9-45e9-8a94-a21684f3f986" (UID: "8a1c4331-d2a9-45e9-8a94-a21684f3f986"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:45.503318 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.503268 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a1c4331-d2a9-45e9-8a94-a21684f3f986-kube-api-access-67w87" (OuterVolumeSpecName: "kube-api-access-67w87") pod "8a1c4331-d2a9-45e9-8a94-a21684f3f986" (UID: "8a1c4331-d2a9-45e9-8a94-a21684f3f986"). InnerVolumeSpecName "kube-api-access-67w87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:50:45.601916 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.601876 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:45.601916 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.601909 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:45.601916 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.601919 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67w87\" (UniqueName: \"kubernetes.io/projected/8a1c4331-d2a9-45e9-8a94-a21684f3f986-kube-api-access-67w87\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:45.601916 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.601927 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a1c4331-d2a9-45e9-8a94-a21684f3f986-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:45.965655 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.965619 2570 generic.go:356] "Generic (PLEG): container finished" podID="8a1c4331-d2a9-45e9-8a94-a21684f3f986" containerID="b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464" exitCode=0 Jul 02 23:50:45.965894 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.965665 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" Jul 02 23:50:45.965894 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.965680 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" event={"ID":"8a1c4331-d2a9-45e9-8a94-a21684f3f986","Type":"ContainerDied","Data":"b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464"} Jul 02 23:50:45.965894 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.965712 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw" event={"ID":"8a1c4331-d2a9-45e9-8a94-a21684f3f986","Type":"ContainerDied","Data":"5c9ddb0d8f62e8510130c2804aebec6a1b2cdbf27af09713d28b907f23083f75"} Jul 02 23:50:45.965894 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.965729 2570 scope.go:117] "RemoveContainer" containerID="b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464" Jul 02 23:50:45.976711 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.976691 2570 scope.go:117] "RemoveContainer" containerID="b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464" Jul 02 23:50:45.976938 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:50:45.976920 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464\": container with ID starting with b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464 not found: ID does not exist" containerID="b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464" Jul 02 23:50:45.976986 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.976947 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464"} err="failed to get container status \"b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464\": rpc error: code = NotFound desc = could not find container \"b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464\": container with ID starting with b7fc98e6b785bdd8bd5882e4822fc7a526fe58982e126f9ca8b8a7dda7abe464 not found: ID does not exist" Jul 02 23:50:45.989483 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.989455 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw"] Jul 02 23:50:45.991502 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:45.991484 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6d35bd/rate-test-model-6d35bd-kserve-8b6857f9f-lj7jw"] Jul 02 23:50:46.008855 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.008802 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" podUID="c9cba7b3-dd57-41ce-a544-0d1327f3aca3" containerName="main" containerID="cri-o://c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64" gracePeriod=45 Jul 02 23:50:46.254552 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.254528 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:50:46.409422 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409369 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-model-cache\") pod \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " Jul 02 23:50:46.409422 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409422 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn592\" (UniqueName: \"kubernetes.io/projected/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-kube-api-access-zn592\") pod \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " Jul 02 23:50:46.409695 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409447 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-dshm\") pod \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " Jul 02 23:50:46.409695 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409468 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-home\") pod \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " Jul 02 23:50:46.409695 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409491 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tmp-dir\") pod \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " Jul 02 23:50:46.409695 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409562 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tls-certs\") pod \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\" (UID: \"c9cba7b3-dd57-41ce-a544-0d1327f3aca3\") " Jul 02 23:50:46.409888 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409733 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-model-cache" (OuterVolumeSpecName: "model-cache") pod "c9cba7b3-dd57-41ce-a544-0d1327f3aca3" (UID: "c9cba7b3-dd57-41ce-a544-0d1327f3aca3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:46.409888 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409797 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-home" (OuterVolumeSpecName: "home") pod "c9cba7b3-dd57-41ce-a544-0d1327f3aca3" (UID: "c9cba7b3-dd57-41ce-a544-0d1327f3aca3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:46.409888 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.409834 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c9cba7b3-dd57-41ce-a544-0d1327f3aca3" (UID: "c9cba7b3-dd57-41ce-a544-0d1327f3aca3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:46.411685 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.411658 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-kube-api-access-zn592" (OuterVolumeSpecName: "kube-api-access-zn592") pod "c9cba7b3-dd57-41ce-a544-0d1327f3aca3" (UID: "c9cba7b3-dd57-41ce-a544-0d1327f3aca3"). InnerVolumeSpecName "kube-api-access-zn592". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 23:50:46.411789 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.411695 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-dshm" (OuterVolumeSpecName: "dshm") pod "c9cba7b3-dd57-41ce-a544-0d1327f3aca3" (UID: "c9cba7b3-dd57-41ce-a544-0d1327f3aca3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 23:50:46.411789 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.411699 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c9cba7b3-dd57-41ce-a544-0d1327f3aca3" (UID: "c9cba7b3-dd57-41ce-a544-0d1327f3aca3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 23:50:46.510635 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.510546 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-model-cache\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:46.510635 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.510576 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn592\" (UniqueName: \"kubernetes.io/projected/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-kube-api-access-zn592\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:46.510635 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.510587 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-dshm\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:46.510635 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.510596 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-home\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:46.510635 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.510604 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tmp-dir\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:46.510635 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.510611 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9cba7b3-dd57-41ce-a544-0d1327f3aca3-tls-certs\") on node \"ip-10-0-136-84.ec2.internal\" DevicePath \"\"" Jul 02 23:50:46.593143 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.593109 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a1c4331-d2a9-45e9-8a94-a21684f3f986" path="/var/lib/kubelet/pods/8a1c4331-d2a9-45e9-8a94-a21684f3f986/volumes" Jul 02 23:50:46.975520 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.975492 2570 generic.go:356] "Generic (PLEG): container finished" podID="c9cba7b3-dd57-41ce-a544-0d1327f3aca3" containerID="c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64" exitCode=0 Jul 02 23:50:46.975715 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.975531 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" Jul 02 23:50:46.975715 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.975541 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" event={"ID":"c9cba7b3-dd57-41ce-a544-0d1327f3aca3","Type":"ContainerDied","Data":"c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64"} Jul 02 23:50:46.975715 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.975571 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45" event={"ID":"c9cba7b3-dd57-41ce-a544-0d1327f3aca3","Type":"ContainerDied","Data":"0610bfa9f7ec81840d73a453beb12059bf05ff9f28495ae24a62b2ff01704058"} Jul 02 23:50:46.975715 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.975588 2570 scope.go:117] "RemoveContainer" containerID="c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64" Jul 02 23:50:46.984557 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.984539 2570 scope.go:117] "RemoveContainer" containerID="c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64" Jul 02 23:50:46.984816 ip-10-0-136-84 kubenswrapper[2570]: E0702 23:50:46.984795 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64\": container with ID starting with c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64 not found: ID does not exist" containerID="c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64" Jul 02 23:50:46.984876 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.984826 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64"} err="failed to get container status \"c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64\": rpc error: code = NotFound desc = could not find container \"c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64\": container with ID starting with c2a47c094e186ea1f00f48a9941350c0d4329435207900cac06dc8f492955f64 not found: ID does not exist" Jul 02 23:50:46.995266 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:46.994844 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45"] Jul 02 23:50:47.001064 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:47.001038 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ed05a5/rate-test-model-ed05a5-kserve-67f964c967-njf45"] Jul 02 23:50:48.592199 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:50:48.592153 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9cba7b3-dd57-41ce-a544-0d1327f3aca3" path="/var/lib/kubelet/pods/c9cba7b3-dd57-41ce-a544-0d1327f3aca3/volumes" Jul 02 23:51:17.802420 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:17.802380 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cjfdn"] Jul 02 23:51:27.791407 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:27.791376 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717265-mpz6k_5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d/cleanup/0.log" Jul 02 23:51:28.037078 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:28.037044 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-895fb_1cf74942-3095-410c-a56f-94d5d136515e/server/0.log" Jul 02 23:51:28.527996 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:28.527966 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79946c878f-zm7zc_7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf/copy-manifests/0.log" Jul 02 23:51:28.535661 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:28.535635 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79946c878f-zm7zc_7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf/manager/0.log" Jul 02 23:51:29.426754 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.426723 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc_33dcec3d-240f-4cde-93dc-dcc5436441e5/util/0.log" Jul 02 23:51:29.434911 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.434883 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc_33dcec3d-240f-4cde-93dc-dcc5436441e5/pull/0.log" Jul 02 23:51:29.442624 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.442599 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc_33dcec3d-240f-4cde-93dc-dcc5436441e5/extract/0.log" Jul 02 23:51:29.558074 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.558040 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k_b3620dde-99de-47d5-a9dd-07b69a05d5d2/util/0.log" Jul 02 23:51:29.568319 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.568295 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k_b3620dde-99de-47d5-a9dd-07b69a05d5d2/pull/0.log" Jul 02 23:51:29.575474 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.575454 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k_b3620dde-99de-47d5-a9dd-07b69a05d5d2/extract/0.log" Jul 02 23:51:29.685552 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.685481 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8_1a584d71-f454-4fa2-b666-2cc56b395866/pull/0.log" Jul 02 23:51:29.692475 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.692454 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8_1a584d71-f454-4fa2-b666-2cc56b395866/extract/0.log" Jul 02 23:51:29.698896 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.698866 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8_1a584d71-f454-4fa2-b666-2cc56b395866/util/0.log" Jul 02 23:51:29.809478 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.809447 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg_cf7c9cda-dad0-414a-9baf-fdae3dfdced0/util/0.log" Jul 02 23:51:29.816699 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.816673 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg_cf7c9cda-dad0-414a-9baf-fdae3dfdced0/pull/0.log" Jul 02 23:51:29.824640 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:29.824617 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg_cf7c9cda-dad0-414a-9baf-fdae3dfdced0/extract/0.log" Jul 02 23:51:30.637014 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:30.636985 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cjfdn_4d5fabed-73af-4192-8ca6-ec1b0d566d95/limitador/0.log" Jul 02 23:51:31.467651 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:31.467621 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86c4d5658d-dgr5d_56af0311-5e2a-4321-a384-c96be5939e49/kube-auth-proxy/0.log" Jul 02 23:51:31.925438 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:31.925406 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d4cf774dc-pdxfl_ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1/router/0.log" Jul 02 23:51:32.703243 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:32.703214 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-xs4hr_162add50-38b2-4d51-b8cf-f665a608dd7b/main/0.log" Jul 02 23:51:32.814564 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:32.814535 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-d2hgn_c078494b-4b9b-447b-8811-5fffd060aa27/main/0.log" Jul 02 23:51:37.007358 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.007314 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2r2d9/must-gather-4rbq5"] Jul 02 23:51:37.007841 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.007797 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1c4331-d2a9-45e9-8a94-a21684f3f986" containerName="main" Jul 02 23:51:37.007841 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.007816 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1c4331-d2a9-45e9-8a94-a21684f3f986" containerName="main" Jul 02 23:51:37.007841 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.007827 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9cba7b3-dd57-41ce-a544-0d1327f3aca3" containerName="main" Jul 02 23:51:37.007841 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.007835 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9cba7b3-dd57-41ce-a544-0d1327f3aca3" containerName="main" Jul 02 23:51:37.008062 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.007952 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1c4331-d2a9-45e9-8a94-a21684f3f986" containerName="main" Jul 02 23:51:37.008062 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.007966 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9cba7b3-dd57-41ce-a544-0d1327f3aca3" containerName="main" Jul 02 23:51:37.011369 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.011346 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2r2d9/must-gather-4rbq5" Jul 02 23:51:37.014178 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.014158 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2r2d9\"/\"kube-root-ca.crt\"" Jul 02 23:51:37.014339 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.014159 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2r2d9\"/\"openshift-service-ca.crt\"" Jul 02 23:51:37.014339 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.014269 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2r2d9\"/\"default-dockercfg-vnk2p\"" Jul 02 23:51:37.021401 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.021377 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2r2d9/must-gather-4rbq5"] Jul 02 23:51:37.151458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.151420 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99-must-gather-output\") pod \"must-gather-4rbq5\" (UID: \"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99\") " pod="openshift-must-gather-2r2d9/must-gather-4rbq5" Jul 02 23:51:37.151458 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.151464 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxlsn\" (UniqueName: \"kubernetes.io/projected/59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99-kube-api-access-nxlsn\") pod \"must-gather-4rbq5\" (UID: \"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99\") " pod="openshift-must-gather-2r2d9/must-gather-4rbq5" Jul 02 23:51:37.252920 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.252885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99-must-gather-output\") pod \"must-gather-4rbq5\" (UID: \"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99\") " pod="openshift-must-gather-2r2d9/must-gather-4rbq5" Jul 02 23:51:37.252920 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.252921 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxlsn\" (UniqueName: \"kubernetes.io/projected/59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99-kube-api-access-nxlsn\") pod \"must-gather-4rbq5\" (UID: \"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99\") " pod="openshift-must-gather-2r2d9/must-gather-4rbq5" Jul 02 23:51:37.253272 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.253253 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99-must-gather-output\") pod \"must-gather-4rbq5\" (UID: \"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99\") " pod="openshift-must-gather-2r2d9/must-gather-4rbq5" Jul 02 23:51:37.261417 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.261354 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxlsn\" (UniqueName: \"kubernetes.io/projected/59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99-kube-api-access-nxlsn\") pod \"must-gather-4rbq5\" (UID: \"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99\") " pod="openshift-must-gather-2r2d9/must-gather-4rbq5" Jul 02 23:51:37.321640 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.321596 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2r2d9/must-gather-4rbq5" Jul 02 23:51:37.456701 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.456671 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2r2d9/must-gather-4rbq5"] Jul 02 23:51:37.458932 ip-10-0-136-84 kubenswrapper[2570]: W0702 23:51:37.458905 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59f132eb_4aa5_4bf1_b1bb_6b72f0d82c99.slice/crio-35020d8b55d4af327207fb895f099e52d19cc144253d005d64eeef57cd16be85 WatchSource:0}: Error finding container 35020d8b55d4af327207fb895f099e52d19cc144253d005d64eeef57cd16be85: Status 404 returned error can't find the container with id 35020d8b55d4af327207fb895f099e52d19cc144253d005d64eeef57cd16be85 Jul 02 23:51:37.461364 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:37.461345 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 23:51:38.246874 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:38.246838 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2r2d9/must-gather-4rbq5" event={"ID":"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99","Type":"ContainerStarted","Data":"35020d8b55d4af327207fb895f099e52d19cc144253d005d64eeef57cd16be85"} Jul 02 23:51:39.255748 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:39.255701 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2r2d9/must-gather-4rbq5" event={"ID":"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99","Type":"ContainerStarted","Data":"1a95cfe8f99f7db9a9c90da070082ed85f0ec65206962fca119193fe3d684eb4"} Jul 02 23:51:39.256336 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:39.256302 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2r2d9/must-gather-4rbq5" event={"ID":"59f132eb-4aa5-4bf1-b1bb-6b72f0d82c99","Type":"ContainerStarted","Data":"33a4aa2bb47d7664a6020fcc03108014a40307ef34cd2fb5a192a2844128f296"} Jul 02 23:51:39.271769 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:39.271700 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2r2d9/must-gather-4rbq5" podStartSLOduration=2.511845544 podStartE2EDuration="3.271679016s" totalImagesPullingTime="759.833472ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:51:39.269935708 +0000 UTC m=+2491.265184032" watchObservedRunningTime="2026-07-02 23:51:39.271679016 +0000 UTC m=+2491.266927309" Jul 02 23:51:39.748288 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:39.748249 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6zz8v_1352ad10-5c46-4abe-ad7e-9c7d3572be46/global-pull-secret-syncer/0.log" Jul 02 23:51:39.957199 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:39.957150 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v9zn5_b96dd76c-b5b4-4324-9dd0-b5bafb90ac0f/konnectivity-agent/0.log" Jul 02 23:51:40.004691 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:40.004594 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-84.ec2.internal_157508bb1f75054b68662588baddc867/haproxy/0.log" Jul 02 23:51:44.460696 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.460608 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc_33dcec3d-240f-4cde-93dc-dcc5436441e5/extract/0.log" Jul 02 23:51:44.484913 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.484885 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc_33dcec3d-240f-4cde-93dc-dcc5436441e5/util/0.log" Jul 02 23:51:44.505655 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.505530 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pvdkc_33dcec3d-240f-4cde-93dc-dcc5436441e5/pull/0.log" Jul 02 23:51:44.531417 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.531390 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k_b3620dde-99de-47d5-a9dd-07b69a05d5d2/extract/0.log" Jul 02 23:51:44.554027 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.553913 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k_b3620dde-99de-47d5-a9dd-07b69a05d5d2/util/0.log" Jul 02 23:51:44.573493 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.573414 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ck46k_b3620dde-99de-47d5-a9dd-07b69a05d5d2/pull/0.log" Jul 02 23:51:44.620431 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.620404 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8_1a584d71-f454-4fa2-b666-2cc56b395866/extract/0.log" Jul 02 23:51:44.641911 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.641879 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8_1a584d71-f454-4fa2-b666-2cc56b395866/util/0.log" Jul 02 23:51:44.665784 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.665753 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73q5wg8_1a584d71-f454-4fa2-b666-2cc56b395866/pull/0.log" Jul 02 23:51:44.696076 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.696038 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg_cf7c9cda-dad0-414a-9baf-fdae3dfdced0/extract/0.log" Jul 02 23:51:44.723065 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.722967 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg_cf7c9cda-dad0-414a-9baf-fdae3dfdced0/util/0.log" Jul 02 23:51:44.750098 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:44.750051 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hqbg_cf7c9cda-dad0-414a-9baf-fdae3dfdced0/pull/0.log" Jul 02 23:51:45.323816 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:45.323788 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cjfdn_4d5fabed-73af-4192-8ca6-ec1b0d566d95/limitador/0.log" Jul 02 23:51:47.048209 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.045459 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/1.log" Jul 02 23:51:47.127673 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.125338 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xq5vp_b735b6c9-3f1e-4d8c-b5fc-a8b4ca5ddca9/cluster-monitoring-operator/0.log" Jul 02 23:51:47.156196 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.156147 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tcjmk_ad411b0b-0075-4295-9a3c-ff0e68d8da8f/kube-state-metrics/0.log" Jul 02 23:51:47.193059 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.193027 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tcjmk_ad411b0b-0075-4295-9a3c-ff0e68d8da8f/kube-rbac-proxy-main/0.log" Jul 02 23:51:47.217927 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.217891 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-tcjmk_ad411b0b-0075-4295-9a3c-ff0e68d8da8f/kube-rbac-proxy-self/0.log" Jul 02 23:51:47.400963 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.400935 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lvgbp_9e544b8e-e401-4b88-ab70-30981b96a3dc/node-exporter/0.log" Jul 02 23:51:47.421111 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.421071 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lvgbp_9e544b8e-e401-4b88-ab70-30981b96a3dc/kube-rbac-proxy/0.log" Jul 02 23:51:47.445971 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.445945 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lvgbp_9e544b8e-e401-4b88-ab70-30981b96a3dc/init-textfile/0.log" Jul 02 23:51:47.822041 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.821991 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-q4z6r_600b96fe-8a50-40ee-94c0-825e04b576ce/prometheus-operator/0.log" Jul 02 23:51:47.841265 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.841233 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-q4z6r_600b96fe-8a50-40ee-94c0-825e04b576ce/kube-rbac-proxy/0.log" Jul 02 23:51:47.867350 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:47.867314 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-n9rnb_5c64aa82-5448-4089-9fd3-9e89dd921a9b/prometheus-operator-admission-webhook/0.log" Jul 02 23:51:50.208061 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.208016 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4"] Jul 02 23:51:50.210965 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.210939 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.221513 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.221488 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4"] Jul 02 23:51:50.297205 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.296430 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-sys\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.297205 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.296486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-podres\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.297205 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.296619 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-proc\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.297205 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.296661 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgw99\" (UniqueName: \"kubernetes.io/projected/bc9ed682-559e-4ce7-8a9d-2741ed11f744-kube-api-access-jgw99\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.297205 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.296707 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-lib-modules\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.397967 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.397913 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-sys\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.397967 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.397967 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-podres\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.398224 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.398036 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-proc\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.398224 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.398065 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgw99\" (UniqueName: \"kubernetes.io/projected/bc9ed682-559e-4ce7-8a9d-2741ed11f744-kube-api-access-jgw99\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.398224 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.398110 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-lib-modules\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.398389 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.398363 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-lib-modules\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.398449 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.398431 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-sys\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.398536 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.398519 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-podres\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.398683 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.398662 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc9ed682-559e-4ce7-8a9d-2741ed11f744-proc\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.407294 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.407264 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgw99\" (UniqueName: \"kubernetes.io/projected/bc9ed682-559e-4ce7-8a9d-2741ed11f744-kube-api-access-jgw99\") pod \"perf-node-gather-daemonset-lpdr4\" (UID: \"bc9ed682-559e-4ce7-8a9d-2741ed11f744\") " pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.462640 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.462554 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78657dfb95-f8cvm_075561db-ccca-4595-9dad-454ed29408e9/console/0.log" Jul 02 23:51:50.496579 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.496543 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-4z9jb_5da6a02f-66c5-4af3-b26f-16fb14e7ff2a/download-server/0.log" Jul 02 23:51:50.524758 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.524706 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:50.711772 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:50.708209 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4"] Jul 02 23:51:51.116515 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:51.116484 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-jbpdf_5aad2a96-277e-4f9c-9f91-dd4eafa2a0b8/volume-data-source-validator/0.log" Jul 02 23:51:51.368840 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:51.368750 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" event={"ID":"bc9ed682-559e-4ce7-8a9d-2741ed11f744","Type":"ContainerStarted","Data":"52d8c35dcf5345766a000b4d6416dafed3e104a6529728dbce7067bcb33bc334"} Jul 02 23:51:51.368840 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:51.368824 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" event={"ID":"bc9ed682-559e-4ce7-8a9d-2741ed11f744","Type":"ContainerStarted","Data":"70d1bf7bf823b54b6d739cdb4ddcbb56f2c052f8fbe4914f1ad5d7780a447a55"} Jul 02 23:51:51.369372 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:51.368856 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:51.387342 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:51.387276 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" podStartSLOduration=1.387253554 podStartE2EDuration="1.387253554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:51:51.383499253 +0000 UTC m=+2503.378747545" watchObservedRunningTime="2026-07-02 23:51:51.387253554 +0000 UTC m=+2503.382501846" Jul 02 23:51:52.133394 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:52.133364 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sgl52_bc6bbd32-8e79-4ee6-b669-dd27240ddd07/dns/0.log" Jul 02 23:51:52.152804 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:52.152778 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sgl52_bc6bbd32-8e79-4ee6-b669-dd27240ddd07/kube-rbac-proxy/0.log" Jul 02 23:51:52.240404 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:52.240375 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mdw9r_5ef83259-01b2-4ae4-bdbc-af8081c9fa8a/dns-node-resolver/0.log" Jul 02 23:51:52.853573 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:52.853545 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fv8lb_55620e5e-72b6-4a11-afd9-f85e0171ba51/node-ca/0.log" Jul 02 23:51:53.976134 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:53.976108 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86c4d5658d-dgr5d_56af0311-5e2a-4321-a384-c96be5939e49/kube-auth-proxy/0.log" Jul 02 23:51:54.126704 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:54.126672 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d4cf774dc-pdxfl_ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1/router/0.log" Jul 02 23:51:54.789683 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:54.789654 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qzpwx_43611282-80ee-4562-bf56-02fb61dff130/serve-healthcheck-canary/0.log" Jul 02 23:51:55.292240 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:55.292196 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bzgbm_acdd4d3d-68f6-427a-9c96-40384b99b3b8/kube-rbac-proxy/0.log" Jul 02 23:51:55.311003 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:55.310968 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bzgbm_acdd4d3d-68f6-427a-9c96-40384b99b3b8/exporter/0.log" Jul 02 23:51:55.330893 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:55.330859 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bzgbm_acdd4d3d-68f6-427a-9c96-40384b99b3b8/extractor/0.log" Jul 02 23:51:57.385048 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:57.385018 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2r2d9/perf-node-gather-daemonset-lpdr4" Jul 02 23:51:57.766392 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:57.766311 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717265-mpz6k_5dfc161d-550e-4d7d-8264-0ecb4fe5fb7d/cleanup/0.log" Jul 02 23:51:57.877243 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:57.877207 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-895fb_1cf74942-3095-410c-a56f-94d5d136515e/server/0.log" Jul 02 23:51:58.098392 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:58.098363 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79946c878f-zm7zc_7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf/manager/0.log" Jul 02 23:51:58.117035 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:58.117010 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79946c878f-zm7zc_7fc8f9e2-9cfc-4c38-a83b-e68a0f76d6bf/copy-manifests/0.log" Jul 02 23:51:59.604727 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:51:59.604695 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6898c65679-qfgbv_f6863319-7673-4582-8e9f-d04bf44b3b50/manager/0.log" Jul 02 23:52:00.882205 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:00.882104 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86c4d5658d-dgr5d_56af0311-5e2a-4321-a384-c96be5939e49/kube-auth-proxy/0.log" Jul 02 23:52:00.995824 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:00.995792 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d4cf774dc-pdxfl_ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1/router/0.log" Jul 02 23:52:02.503932 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:02.503896 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86c4d5658d-dgr5d_56af0311-5e2a-4321-a384-c96be5939e49/kube-auth-proxy/0.log" Jul 02 23:52:02.633712 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:02.633679 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d4cf774dc-pdxfl_ff0fad6c-5cdb-4e12-9fa8-305bdd3466e1/router/0.log" Jul 02 23:52:05.108123 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:05.108089 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-kb5t6_2384cb4b-da67-4522-8ca7-d721551a31bb/kube-storage-version-migrator-operator/1.log" Jul 02 23:52:05.108938 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:05.108918 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-kb5t6_2384cb4b-da67-4522-8ca7-d721551a31bb/kube-storage-version-migrator-operator/0.log" Jul 02 23:52:06.080788 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.080761 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qnb6_9c2e3cc3-5a74-4577-81ad-4439993bfb17/kube-multus-additional-cni-plugins/0.log" Jul 02 23:52:06.100879 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.100856 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qnb6_9c2e3cc3-5a74-4577-81ad-4439993bfb17/egress-router-binary-copy/0.log" Jul 02 23:52:06.122136 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.122102 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qnb6_9c2e3cc3-5a74-4577-81ad-4439993bfb17/cni-plugins/0.log" Jul 02 23:52:06.142081 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.142044 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qnb6_9c2e3cc3-5a74-4577-81ad-4439993bfb17/bond-cni-plugin/0.log" Jul 02 23:52:06.161573 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.161543 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qnb6_9c2e3cc3-5a74-4577-81ad-4439993bfb17/routeoverride-cni/0.log" Jul 02 23:52:06.183377 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.183345 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qnb6_9c2e3cc3-5a74-4577-81ad-4439993bfb17/whereabouts-cni-bincopy/0.log" Jul 02 23:52:06.202545 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.202519 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qnb6_9c2e3cc3-5a74-4577-81ad-4439993bfb17/whereabouts-cni/0.log" Jul 02 23:52:06.574104 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.574066 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rrq2p_61673e8e-ad7f-482f-8b58-2dfdb5d81ddc/kube-multus/0.log" Jul 02 23:52:06.623694 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.623667 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4spv8_84f0d6e1-770c-4014-a4dc-7d27e5a7b1db/network-metrics-daemon/0.log" Jul 02 23:52:06.641138 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:06.641101 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4spv8_84f0d6e1-770c-4014-a4dc-7d27e5a7b1db/kube-rbac-proxy/0.log" Jul 02 23:52:07.523122 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.523087 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-controller/0.log" Jul 02 23:52:07.538885 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.538850 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/1.log" Jul 02 23:52:07.539034 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.538964 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovn-acl-logging/0.log" Jul 02 23:52:07.555566 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.555486 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/kube-rbac-proxy-node/0.log" Jul 02 23:52:07.574666 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.574633 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/kube-rbac-proxy-ovn-metrics/0.log" Jul 02 23:52:07.592577 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.592547 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/northd/0.log" Jul 02 23:52:07.611656 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.611631 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/nbdb/0.log" Jul 02 23:52:07.630565 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.630536 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/sbdb/0.log" Jul 02 23:52:07.681997 ip-10-0-136-84 kubenswrapper[2570]: I0702 23:52:07.681966 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g6gs8_3dca5a94-9f5a-4269-b9b6-5394393fa89f/ovnkube-controller/0.log"