Sep 17 17:38:38.390720 ip-10-0-138-30 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 17 17:38:38.390986 ip-10-0-138-30 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 17 17:38:38.391084 ip-10-0-138-30 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 17 17:38:38.391697 ip-10-0-138-30 systemd[1]: Failed to start Kubernetes Kubelet. Sep 17 17:38:48.611890 ip-10-0-138-30 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 17 17:38:48.611912 ip-10-0-138-30 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0581118806044b7989d1be80e21e819a -- Sep 17 17:40:54.353959 ip-10-0-138-30 systemd[1]: Starting Kubernetes Kubelet... Sep 17 17:40:57.817282 ip-10-0-138-30 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 17:40:57.817282 ip-10-0-138-30 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 17:40:57.817282 ip-10-0-138-30 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 17:40:57.817282 ip-10-0-138-30 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 17:40:57.817282 ip-10-0-138-30 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 17:40:57.818003 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.817364 2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 17:40:57.821806 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821783 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:40:57.821806 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821806 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821810 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821814 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821817 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821820 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821823 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821826 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821829 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821833 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821837 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821840 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821843 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821846 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821849 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821851 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821855 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821857 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821860 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821869 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:40:57.821877 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821872 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821875 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821877 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821880 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821882 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821885 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821888 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821891 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821894 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821897 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821899 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821902 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821904 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821907 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821909 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821911 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821914 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821916 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821919 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821921 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:40:57.822389 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821923 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821926 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821928 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821930 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821933 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821936 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821938 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821940 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821943 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821945 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821947 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821949 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821953 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821955 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821958 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821960 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821963 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821966 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821969 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821973 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:40:57.822894 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821977 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821980 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821982 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821984 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821987 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821989 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821992 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821994 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821997 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.821999 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822002 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822005 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822007 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822009 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822013 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822016 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822020 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822023 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822026 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:40:57.823369 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822028 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822031 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822033 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822036 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822038 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822041 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822045 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822048 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822050 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822053 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822055 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822058 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822060 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822063 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822065 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822068 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822070 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822073 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822076 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822078 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:40:57.823840 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822839 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822849 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822853 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822856 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822859 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822862 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822864 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822867 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822869 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822872 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822874 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822877 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822879 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822882 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822884 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822887 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822890 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822892 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822895 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:40:57.824488 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822899 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822902 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822905 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822907 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822910 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822913 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822916 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822918 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822922 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822925 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822928 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822930 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822933 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822935 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822938 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822940 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822943 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822945 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822948 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822950 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:40:57.825190 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822953 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822955 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822958 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822961 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822963 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822965 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822968 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822970 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822973 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822975 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822978 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822981 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822983 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822986 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822989 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822991 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822994 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822996 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.822999 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823001 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:40:57.825962 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823003 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823006 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823008 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823010 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823013 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823015 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823018 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823020 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823022 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823025 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823027 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823031 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823033 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823035 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823038 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823040 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823042 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823045 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823047 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823050 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:40:57.826828 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823052 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823055 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823058 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823060 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823064 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823067 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823070 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823072 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823075 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823077 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823081 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823084 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823087 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823089 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823092 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823094 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823096 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823099 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823101 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:40:57.827323 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823103 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823477 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823482 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823484 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823487 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823490 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823492 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823495 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823497 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823499 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823502 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823504 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823507 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823510 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823512 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823515 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823518 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823520 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823523 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823526 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:40:57.827916 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823529 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823532 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823534 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823537 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823540 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823542 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823544 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823547 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823549 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823552 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823554 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823557 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823559 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823561 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823565 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823569 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823572 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823575 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823577 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823580 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:40:57.828428 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823582 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823585 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823587 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823590 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823592 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823595 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823597 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823600 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823602 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823605 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823607 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823610 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823629 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823645 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823648 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823650 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823653 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823656 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823658 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823661 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:40:57.829151 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823663 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823666 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823669 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823671 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823674 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823676 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823679 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823681 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823684 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823686 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823689 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823691 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823694 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823696 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823698 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823702 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823704 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823706 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823709 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823711 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823714 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:40:57.829676 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823717 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823719 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823722 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823726 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823730 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823733 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823735 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823738 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823741 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823743 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823746 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823748 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823750 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823753 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823755 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823757 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823760 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823764 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.823767 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:40:57.830240 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823852 2586 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823889 2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823897 2586 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823902 2586 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823906 2586 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823909 2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823914 2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823918 2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823921 2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823925 2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823928 2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823932 2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823936 2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823940 2586 flags.go:64] FLAG: --cgroup-root="" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823943 2586 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823946 2586 flags.go:64] FLAG: --client-ca-file="" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823949 2586 flags.go:64] FLAG: --cloud-provider="external" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823952 2586 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823957 2586 flags.go:64] FLAG: --cluster-domain="" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823960 2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823963 2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823966 2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823970 2586 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823973 2586 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 17:40:57.830799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823976 2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823979 2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823982 2586 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823985 2586 flags.go:64] FLAG: --contention-profiling="false" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823988 2586 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823991 2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823995 2586 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.823997 2586 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.824002 2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.824005 2586 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.824008 2586 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.824012 2586 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.824016 2586 flags.go:64] FLAG: --enable-server="true" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.824019 2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825259 2586 flags.go:64] FLAG: --event-burst="100" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825268 2586 flags.go:64] FLAG: --event-qps="50" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825281 2586 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825285 2586 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825289 2586 flags.go:64] FLAG: --eviction-hard="" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825295 2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825299 2586 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825303 2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825309 2586 flags.go:64] FLAG: --eviction-soft="" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825315 2586 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825326 2586 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 17:40:57.831379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825333 2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825338 2586 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825357 2586 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825362 2586 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825368 2586 flags.go:64] FLAG: --feature-gates="" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825375 2586 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825381 2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825386 2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825396 2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825402 2586 flags.go:64] FLAG: --healthz-port="10248" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825407 2586 flags.go:64] FLAG: --help="false" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825411 2586 flags.go:64] FLAG: --hostname-override="ip-10-0-138-30.ec2.internal" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825417 2586 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825422 2586 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825427 2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825433 2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825444 2586 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825449 2586 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825454 2586 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825458 2586 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825463 2586 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825468 2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825474 2586 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825479 2586 flags.go:64] FLAG: --kube-reserved="" Sep 17 17:40:57.832032 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825484 2586 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825494 2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825500 2586 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825505 2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825510 2586 flags.go:64] FLAG: --lock-file="" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825514 2586 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825520 2586 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825525 2586 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825534 2586 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825543 2586 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825549 2586 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825560 2586 flags.go:64] FLAG: --logging-format="text" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825710 2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825729 2586 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825735 2586 flags.go:64] FLAG: --manifest-url="" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825741 2586 flags.go:64] FLAG: --manifest-url-header="" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825763 2586 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825769 2586 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825777 2586 flags.go:64] FLAG: --max-pods="110" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825782 2586 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825788 2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825792 2586 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825797 2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825803 2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825808 2586 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 17:40:57.832639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825813 2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825829 2586 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825833 2586 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825839 2586 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825844 2586 flags.go:64] FLAG: --pod-cidr="" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825849 2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825859 2586 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825863 2586 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825870 2586 flags.go:64] FLAG: --pods-per-core="0" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825874 2586 flags.go:64] FLAG: --port="10250" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825879 2586 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825884 2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08cb7a27279cf06d0" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825890 2586 flags.go:64] FLAG: --qos-reserved="" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825895 2586 flags.go:64] FLAG: --read-only-port="10255" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825900 2586 flags.go:64] FLAG: --register-node="true" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825905 2586 flags.go:64] FLAG: --register-with-taints="" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825911 2586 flags.go:64] FLAG: --registry-burst="10" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825916 2586 flags.go:64] FLAG: --registry-qps="5" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825921 2586 flags.go:64] FLAG: --reserved-cpus="" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825925 2586 flags.go:64] FLAG: --reserved-memory="" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825933 2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825938 2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825943 2586 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825948 2586 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825954 2586 flags.go:64] FLAG: --runonce="false" Sep 17 17:40:57.833270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825959 2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825964 2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825969 2586 flags.go:64] FLAG: --seccomp-default="false" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825973 2586 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825977 2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825982 2586 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825987 2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825992 2586 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.825997 2586 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826002 2586 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826007 2586 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826012 2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826017 2586 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826021 2586 flags.go:64] FLAG: --system-cgroups="" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826026 2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826036 2586 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826041 2586 flags.go:64] FLAG: --tls-cert-file="" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826046 2586 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826053 2586 flags.go:64] FLAG: --tls-min-version="" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826058 2586 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826063 2586 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826067 2586 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826073 2586 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826078 2586 flags.go:64] FLAG: --v="2" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826085 2586 flags.go:64] FLAG: --version="false" Sep 17 17:40:57.833991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826091 2586 flags.go:64] FLAG: --vmodule="" Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826097 2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.826103 2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826303 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826313 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826319 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826324 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826328 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826333 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826337 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826341 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826345 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826350 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826354 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826358 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826363 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826367 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826371 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826375 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826379 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:40:57.834642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826383 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826388 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826393 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826399 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826405 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826410 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826416 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826420 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826425 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826429 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826434 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826438 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826442 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826446 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826450 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826457 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826461 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826465 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826469 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:40:57.835144 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826475 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826481 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826486 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826490 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826494 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826499 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826503 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826508 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826512 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826516 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826521 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826526 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826530 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826535 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826539 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826543 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826548 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826552 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826556 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826562 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:40:57.835642 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826566 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826571 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826574 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826579 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826583 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826587 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826592 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826596 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826601 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826605 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826609 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826635 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826640 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826646 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826650 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826655 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826659 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826663 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826666 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:40:57.836137 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826670 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826674 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826678 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826683 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826687 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826691 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826695 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826700 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826704 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826708 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826712 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826716 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826723 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826727 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826731 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826735 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826739 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826743 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826747 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826751 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:40:57.836608 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826755 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826761 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826768 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.826774 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.827376 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.835030 2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.835050 2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.835072 2586 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.835079 2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835137 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835142 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835145 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835148 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835153 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:40:57.837117 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835157 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835160 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835164 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835166 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835169 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835173 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835177 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835180 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835183 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835186 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835190 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835194 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835197 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835200 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835203 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835206 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835208 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835211 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835214 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:40:57.837480 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835216 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835219 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835221 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835224 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835226 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835231 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835234 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835237 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835239 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835242 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835244 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835247 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835249 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835252 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835255 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835257 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835260 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835262 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835264 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835267 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:40:57.837981 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835269 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835272 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835274 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835277 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835280 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835282 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835285 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835287 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835290 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835292 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835295 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835297 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835300 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835302 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835305 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835307 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835310 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835313 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835316 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835318 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:40:57.838474 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835321 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835324 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835326 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835328 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835331 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835333 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835336 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835339 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835341 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835344 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835346 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835349 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835351 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835354 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835356 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835359 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835361 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835364 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835366 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835369 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:40:57.839011 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835371 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835373 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835376 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835378 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835381 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835383 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835386 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835388 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835391 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835394 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835397 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835400 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835402 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835405 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835407 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:40:57.839489 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.835413 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835525 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835531 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835534 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835538 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835540 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835543 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835546 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835549 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835552 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835554 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835557 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835559 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835562 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835564 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835567 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835569 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835572 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835574 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 17:40:57.839904 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835577 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835579 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835582 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835584 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835587 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835589 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835592 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835594 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835597 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835600 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835603 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835606 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835609 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835611 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835628 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835631 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835633 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835636 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835638 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835641 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 17:40:57.840392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835643 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835646 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835649 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835651 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835654 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835657 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835659 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835662 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835664 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835667 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835669 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835671 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835674 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835676 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835679 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835681 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835684 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835686 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835689 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835691 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 17:40:57.840944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835694 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835697 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835700 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835703 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835705 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835708 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835711 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835713 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835716 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835718 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835721 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835723 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835726 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835728 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835730 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835733 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835735 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835740 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835743 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835745 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 17:40:57.841426 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835748 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835751 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835753 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835756 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835758 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835760 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835763 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835770 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835773 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835776 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835778 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835781 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835784 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835787 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835791 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835793 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835796 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835799 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835802 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 17:40:57.841929 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835804 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 17:40:57.842392 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:57.835807 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 17:40:57.842392 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.835812 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 17:40:57.842392 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.835951 2586 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 17:40:57.842392 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.838267 2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 17:40:57.842392 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.838965 2586 server.go:1019] "Starting client certificate rotation" Sep 17 17:40:57.842392 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.839066 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 17:40:57.842392 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.839112 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 17:40:57.842392 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.841176 2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 17:40:57.844501 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.844477 2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 17:40:57.854889 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.854866 2586 log.go:25] "Validated CRI v1 runtime API" Sep 17 17:40:57.857009 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.856992 2586 log.go:25] "Validated CRI v1 image API" Sep 17 17:40:57.857786 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.857763 2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 17:40:57.860954 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.860919 2586 fs.go:135] Filesystem UUIDs: map[308cfa86-e423-4492-a1e1-8bfddd6b4e88:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d1575bd0-6a87-4caa-8d8d-4129fd2cc39d:/dev/nvme0n1p3] Sep 17 17:40:57.861022 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.860956 2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 17:40:57.865309 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.865183 2586 manager.go:216] Machine: {Timestamp:2026-09-17 17:40:57.864694791 +0000 UTC m=+2.013842203 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100336 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22fa2a615a077395654021c3379945 SystemUUID:ec22fa2a-615a-0773-9565-4021c3379945 BootID:05811188-0604-4b79-89d1-be80e21e819a Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e5:f4:0b:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e5:f4:0b:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:a8:11:ef:5d:0c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 17:40:57.865309 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.865299 2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 17:40:57.865417 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.865384 2586 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 17:40:57.866808 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.866778 2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 17:40:57.866949 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.866810 2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-30.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 17:40:57.868012 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.867998 2586 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 17:40:57.868067 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.868014 2586 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 17:40:57.868067 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.868024 2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 17:40:57.868067 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.868035 2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 17:40:57.868195 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.868114 2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 17:40:57.871227 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871212 2586 state_mem.go:36] "Initialized new in-memory state store" Sep 17 17:40:57.871356 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871346 2586 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 17:40:57.871414 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871381 2586 kubelet.go:486] "Attempting to sync node with API server" Sep 17 17:40:57.871414 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871393 2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 17:40:57.871414 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871407 2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 17:40:57.871519 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871417 2586 kubelet.go:398] "Adding apiserver pod source" Sep 17 17:40:57.871519 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871426 2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 17:40:57.871835 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871822 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 17:40:57.871899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.871843 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 17:40:57.873849 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.873834 2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 17 17:40:57.876329 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.876312 2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 17:40:57.876372 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.876346 2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 17:40:57.877680 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877667 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 17:40:57.877731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877685 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 17:40:57.877731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877692 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 17:40:57.877731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877697 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 17:40:57.877731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877710 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 17:40:57.877731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877719 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 17:40:57.877731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877733 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 17:40:57.877899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877738 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 17:40:57.877899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877745 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 17:40:57.877899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877750 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 17:40:57.877899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877765 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 17:40:57.877899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877770 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 17:40:57.877899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877801 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 17:40:57.877899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.877810 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 17:40:57.881676 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.881660 2586 server.go:1268] "Started kubelet" Sep 17 17:40:57.881791 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.881727 2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 17:40:57.881842 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.881772 2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 17:40:57.881901 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.881844 2586 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 17:40:57.882649 ip-10-0-138-30 systemd[1]: Started Kubernetes Kubelet. Sep 17 17:40:57.883200 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.882904 2586 server.go:310] "Adding debug handlers to kubelet server" Sep 17 17:40:57.883322 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.883305 2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 17:40:57.884823 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.884805 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 17:40:57.911463 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.911391 2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 17:40:57.912274 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.912246 2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-30.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 17:40:57.912556 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.912532 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 17:40:57.912708 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.912563 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 17:40:57.917340 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.917321 2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 17:40:57.917433 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.917350 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 17:40:57.917634 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.917451 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:57.917888 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.917837 2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 17:40:57.917888 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.917876 2586 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 17:40:57.917888 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.917890 2586 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 17:40:57.918065 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918052 2586 reconstruct.go:97] "Volume reconstruction finished" Sep 17 17:40:57.918107 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918066 2586 reconciler.go:29] "Reconciler: start to sync state" Sep 17 17:40:57.918394 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918372 2586 factory.go:55] Registering systemd factory Sep 17 17:40:57.918394 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918394 2586 factory.go:223] Registration of the systemd container factory successfully Sep 17 17:40:57.918767 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918753 2586 factory.go:153] Registering CRI-O factory Sep 17 17:40:57.918843 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918770 2586 factory.go:223] Registration of the crio container factory successfully Sep 17 17:40:57.918843 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918825 2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 17:40:57.918942 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918844 2586 factory.go:103] Registering Raw factory Sep 17 17:40:57.918942 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.918892 2586 manager.go:1196] Started watching for new ooms in manager Sep 17 17:40:57.919450 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.919435 2586 manager.go:349] Starting recovery of all containers Sep 17 17:40:57.925833 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.925780 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 17:40:57.925975 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.925909 2586 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 17:40:57.926967 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.925712 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-30.ec2.internal.18d62c8672529c7d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-30.ec2.internal,UID:ip-10-0-138-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-30.ec2.internal,},FirstTimestamp:2026-09-17 17:40:57.881607293 +0000 UTC m=+2.030754704,LastTimestamp:2026-09-17 17:40:57.881607293 +0000 UTC m=+2.030754704,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-30.ec2.internal,}" Sep 17 17:40:57.930506 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.930484 2586 manager.go:354] Recovery completed Sep 17 17:40:57.934930 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.934913 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:40:57.937064 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.937039 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:40:57.937170 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.937081 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:40:57.937170 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.937093 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:40:57.937707 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.937687 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f2kpn" Sep 17 17:40:57.937800 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.937720 2586 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 17:40:57.937800 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.937730 2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 17:40:57.937800 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.937746 2586 state_mem.go:36] "Initialized new in-memory state store" Sep 17 17:40:57.937930 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.937723 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:40:57.938556 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.938542 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:40:57.938657 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.938572 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:40:57.938657 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.938586 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:40:57.938898 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.938881 2586 policy_none.go:49] "None policy: Start" Sep 17 17:40:57.938945 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.938903 2586 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 17:40:57.938945 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.938915 2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 17:40:57.938945 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.938851 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-30.ec2.internal.18d62c8675a0d46d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-30.ec2.internal,UID:ip-10-0-138-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-30.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-30.ec2.internal,},FirstTimestamp:2026-09-17 17:40:57.937065069 +0000 UTC m=+2.086212479,LastTimestamp:2026-09-17 17:40:57.937065069 +0000 UTC m=+2.086212479,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-30.ec2.internal,}" Sep 17 17:40:57.941812 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.941800 2586 policy_none.go:47] "Start" Sep 17 17:40:57.945819 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.945798 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f2kpn" Sep 17 17:40:57.951818 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.951725 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-30.ec2.internal.18d62c8675a12945 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-30.ec2.internal,UID:ip-10-0-138-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-30.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-30.ec2.internal,},FirstTimestamp:2026-09-17 17:40:57.937086789 +0000 UTC m=+2.086234203,LastTimestamp:2026-09-17 17:40:57.937086789 +0000 UTC m=+2.086234203,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-30.ec2.internal,}" Sep 17 17:40:57.988649 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.988632 2586 manager.go:337] "Starting Device Plugin manager" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.988683 2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.988695 2586 server.go:87] "Starting device plugin registration server" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.988979 2586 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.989003 2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.989347 2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.989439 2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:57.989449 2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.990340 2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 17:40:58.000699 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:57.990448 2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.030711 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.030673 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 17:40:58.032117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.032097 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 17:40:58.032211 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.032164 2586 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 17:40:58.032211 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.032193 2586 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 17:40:58.032307 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.032241 2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 17:40:58.034705 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.034680 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:40:58.090112 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.090014 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:40:58.091126 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.091107 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:40:58.091212 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.091148 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:40:58.091212 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.091161 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:40:58.091212 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.091185 2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.101351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.101331 2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.101434 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.101359 2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-30.ec2.internal\": node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.129804 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.129766 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.132715 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.132694 2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal"] Sep 17 17:40:58.132785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.132774 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:40:58.133728 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.133704 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:40:58.133845 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.133739 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:40:58.133845 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.133754 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:40:58.134857 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.134845 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:40:58.134979 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.134964 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.135054 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.134994 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:40:58.135664 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.135646 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:40:58.135754 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.135676 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:40:58.135754 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.135698 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:40:58.135754 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.135714 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:40:58.135754 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.135735 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:40:58.135754 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.135745 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:40:58.136813 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.136795 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.136909 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.136828 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:40:58.137561 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.137545 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:40:58.137656 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.137573 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:40:58.137656 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.137582 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:40:58.161485 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.161457 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-30.ec2.internal\" not found" node="ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.166002 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.165984 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-30.ec2.internal\" not found" node="ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.220221 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.220188 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2776e3fe0741747d431c1bf536cbf4b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal\" (UID: \"2776e3fe0741747d431c1bf536cbf4b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.220379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.220229 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2776e3fe0741747d431c1bf536cbf4b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal\" (UID: \"2776e3fe0741747d431c1bf536cbf4b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.220379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.220255 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/06841d19bfd80202ae04f77c7199dad4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-30.ec2.internal\" (UID: \"06841d19bfd80202ae04f77c7199dad4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.230498 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.230472 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.321308 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.321274 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2776e3fe0741747d431c1bf536cbf4b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal\" (UID: \"2776e3fe0741747d431c1bf536cbf4b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.321462 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.321244 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2776e3fe0741747d431c1bf536cbf4b1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal\" (UID: \"2776e3fe0741747d431c1bf536cbf4b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.321462 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.321364 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2776e3fe0741747d431c1bf536cbf4b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal\" (UID: \"2776e3fe0741747d431c1bf536cbf4b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.321462 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.321385 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/06841d19bfd80202ae04f77c7199dad4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-30.ec2.internal\" (UID: \"06841d19bfd80202ae04f77c7199dad4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.321462 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.321411 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/06841d19bfd80202ae04f77c7199dad4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-30.ec2.internal\" (UID: \"06841d19bfd80202ae04f77c7199dad4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.321462 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.321437 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2776e3fe0741747d431c1bf536cbf4b1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal\" (UID: \"2776e3fe0741747d431c1bf536cbf4b1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.331384 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.331356 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.432244 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.432215 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.465426 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.465386 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.470184 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.469961 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" Sep 17 17:40:58.532739 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.532692 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.633227 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.633194 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.733762 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.733682 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.834322 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.834282 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.839446 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.839426 2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 17:40:58.839608 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.839585 2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 17:40:58.935388 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:58.935353 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:58.951564 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.951521 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 17:35:57 +0000 UTC" deadline="2028-03-05 13:52:12.614128971 +0000 UTC" Sep 17 17:40:58.951564 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:58.951561 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12836h11m13.662572253s" Sep 17 17:40:59.036359 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:59.036285 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:59.093283 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:59.093234 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06841d19bfd80202ae04f77c7199dad4.slice/crio-8445384a42da506527563ad81052828393a3abbb97992db9d51dbd275f4b7d4b WatchSource:0}: Error finding container 8445384a42da506527563ad81052828393a3abbb97992db9d51dbd275f4b7d4b: Status 404 returned error can't find the container with id 8445384a42da506527563ad81052828393a3abbb97992db9d51dbd275f4b7d4b Sep 17 17:40:59.093545 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:40:59.093524 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2776e3fe0741747d431c1bf536cbf4b1.slice/crio-918f4c81fc8acbea99768ed5d6a1857e2c6a3b43328d1057720e93e733669e2d WatchSource:0}: Error finding container 918f4c81fc8acbea99768ed5d6a1857e2c6a3b43328d1057720e93e733669e2d: Status 404 returned error can't find the container with id 918f4c81fc8acbea99768ed5d6a1857e2c6a3b43328d1057720e93e733669e2d Sep 17 17:40:59.098374 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.098357 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 17:40:59.130202 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.130170 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 17:40:59.131365 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.131344 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientMemory" Sep 17 17:40:59.131469 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.131376 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 17:40:59.131469 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.131389 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeHasSufficientPID" Sep 17 17:40:59.137118 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:59.137099 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:59.237894 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:59.237857 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:59.273267 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.273241 2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:40:59.338685 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:59.338571 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-30.ec2.internal\" not found" Sep 17 17:40:59.422099 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.422068 2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:40:59.430130 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.430104 2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:40:59.518411 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.518348 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" Sep 17 17:40:59.533219 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.533186 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 17:40:59.534474 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.534250 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" Sep 17 17:40:59.545826 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.545794 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 17:40:59.872494 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.872455 2586 apiserver.go:52] "Watching apiserver" Sep 17 17:40:59.878239 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.878211 2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 17:40:59.878738 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.878705 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-ndfck","openshift-image-registry/node-ca-tnn9m","openshift-multus/multus-additional-cni-plugins-pcvrb","openshift-network-diagnostics/network-check-target-wf9fj","openshift-network-operator/iptables-alerter-8spf2","kube-system/global-pull-secret-syncer-xs9vn","kube-system/konnectivity-agent-mz7nt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4","openshift-dns/node-resolver-kz957","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal","openshift-multus/multus-v56s5","openshift-multus/network-metrics-daemon-xnrfz","openshift-ovn-kubernetes/ovnkube-node-6mvdd","kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal"] Sep 17 17:40:59.880289 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.880270 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:40:59.881762 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.881738 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.884503 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.883546 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:40:59.884757 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.884742 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:40:59.884833 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:59.884815 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:40:59.886054 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.886032 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 17:40:59.886227 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.886214 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 17:40:59.886366 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.886353 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 17:40:59.886681 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.886666 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 17:40:59.886843 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.886830 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 17:40:59.888529 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.888510 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:40:59.889091 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.889073 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 17:40:59.889726 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.889701 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xsvs4\"" Sep 17 17:40:59.889928 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.889906 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 17:40:59.892088 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.890864 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gvbv4\"" Sep 17 17:40:59.892088 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.891267 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 17:40:59.892088 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.891340 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 17:40:59.893230 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.893212 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 17:40:59.893935 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.893542 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-flss9\"" Sep 17 17:40:59.893935 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.893808 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 17:40:59.894140 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.894123 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 17:40:59.894439 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.894421 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:40:59.894717 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.894700 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:40:59.894815 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:59.894767 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:40:59.894960 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.894942 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 17:40:59.896271 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.896064 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 17:40:59.896271 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.896124 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 17:40:59.896410 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.896380 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5nz9z\"" Sep 17 17:40:59.896733 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.896713 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kz957" Sep 17 17:40:59.898366 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.898344 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.899571 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.899551 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dmw27\"" Sep 17 17:40:59.899672 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.899588 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 17:40:59.899845 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.899832 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 17:40:59.899845 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.899840 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:40:59.900245 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.900227 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hxzvp\"" Sep 17 17:40:59.904122 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.901189 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:40:59.904122 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.902822 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 17:40:59.904122 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.903018 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:40:59.904122 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.903257 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 17:40:59.904122 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.903459 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 17:40:59.904122 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.903932 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 17:40:59.904384 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.904139 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:40:59.904384 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.904196 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.904384 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:40:59.904264 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:40:59.904384 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.904273 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 17:40:59.905103 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.905083 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6l7ct\"" Sep 17 17:40:59.905331 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.905312 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nb8hq\"" Sep 17 17:40:59.905672 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.905655 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 17:40:59.905894 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.905857 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 17:40:59.906352 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.906331 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 17:40:59.906435 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.906340 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wfmdv\"" Sep 17 17:40:59.918728 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.918702 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 17:40:59.919264 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.919243 2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 17:40:59.930894 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.930867 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-cni-netd\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.931014 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.930918 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.931014 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.930948 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/720b09de-1c22-4a55-88f6-66aca41add21-ovn-node-metrics-cert\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.931014 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.930974 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th9mr\" (UniqueName: \"kubernetes.io/projected/720b09de-1c22-4a55-88f6-66aca41add21-kube-api-access-th9mr\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.931014 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.930997 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-systemd\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.931192 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931030 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-hostroot\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.931192 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931056 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.931192 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931081 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-kubernetes\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.931192 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931100 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-var-lib-kubelet\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.931192 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931116 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86df2dae-c4b8-4042-a75a-29e68a739826-multus-daemon-config\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.931192 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931147 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:40:59.931192 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931176 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-sys\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931198 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-lib-modules\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931224 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-tuned\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931245 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-cnibin\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931263 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931307 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-socket-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931340 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931365 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-modprobe-d\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931404 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-cni-multus\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931427 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t6fw\" (UniqueName: \"kubernetes.io/projected/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-kube-api-access-9t6fw\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:40:59.931456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931447 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysconfig\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.931815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931469 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-cni-bin\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.931815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931493 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-etc-kubernetes\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.931815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931528 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ctbf\" (UniqueName: \"kubernetes.io/projected/86df2dae-c4b8-4042-a75a-29e68a739826-kube-api-access-5ctbf\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.931815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931556 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-var-lib-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.931815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931570 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-log-socket\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.931815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931605 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3520ab4e-6be6-42fb-888f-cbd57c6e7506-konnectivity-ca\") pod \"konnectivity-agent-mz7nt\" (UID: \"3520ab4e-6be6-42fb-888f-cbd57c6e7506\") " pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:40:59.931815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931644 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-multus-certs\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.931815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931676 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-systemd-units\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931697 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-systemd\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931862 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-cni-bin\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931890 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc4fz\" (UniqueName: \"kubernetes.io/projected/9eedb140-05c7-48ce-8171-13998ef8be9e-kube-api-access-vc4fz\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931923 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d3e728a9-fb95-4223-9920-37a2972a101c-kubelet-config\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931940 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931937 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42qnm\" (UniqueName: \"kubernetes.io/projected/7191de83-8ab8-4a34-91ec-2bc5fb95368b-kube-api-access-42qnm\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.931987 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-run-netns\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932001 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-ovnkube-config\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932015 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-host\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932029 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whtm9\" (UniqueName: \"kubernetes.io/projected/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-kube-api-access-whtm9\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932041 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-run\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932060 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-host\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932084 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932100 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-env-overrides\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932115 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-serviceca\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932129 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7191de83-8ab8-4a34-91ec-2bc5fb95368b-iptables-alerter-script\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:40:59.932182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932150 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7191de83-8ab8-4a34-91ec-2bc5fb95368b-host-slash\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932172 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d0ec105-91bf-460a-9fcf-307246b66218-hosts-file\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932197 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysctl-d\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932218 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-etc-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932241 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-ovnkube-script-lib\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932258 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932271 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-sys-fs\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932285 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-socket-dir-parent\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932301 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932322 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp9hf\" (UniqueName: \"kubernetes.io/projected/4d0ec105-91bf-460a-9fcf-307246b66218-kube-api-access-jp9hf\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932356 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-device-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932387 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3520ab4e-6be6-42fb-888f-cbd57c6e7506-agent-certs\") pod \"konnectivity-agent-mz7nt\" (UID: \"3520ab4e-6be6-42fb-888f-cbd57c6e7506\") " pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932412 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-tmp\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932436 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2np97\" (UniqueName: \"kubernetes.io/projected/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-kube-api-access-2np97\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932475 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-kubelet\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.932983 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.932510 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d3e728a9-fb95-4223-9920-37a2972a101c-dbus\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:40:59.936701 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936672 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86df2dae-c4b8-4042-a75a-29e68a739826-cni-binary-copy\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.936809 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936713 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-netns\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.936809 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936742 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-node-log\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.936809 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936766 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:40:59.936809 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936792 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:40:59.937031 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936817 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-cni-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.937031 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936853 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-k8s-cni-cncf-io\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.937031 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936879 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-conf-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.937031 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936896 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcwrn\" (UniqueName: \"kubernetes.io/projected/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-kube-api-access-gcwrn\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:40:59.937031 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936918 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-system-cni-dir\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:40:59.937031 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.936969 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-registration-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:40:59.937031 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937011 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:40:59.937351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937036 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysctl-conf\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:40:59.937351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937059 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-system-cni-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.937351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937082 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-ovn\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.937351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937105 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-run-ovn-kubernetes\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.937351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937127 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-cnibin\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:40:59.937351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937153 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d0ec105-91bf-460a-9fcf-307246b66218-tmp-dir\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:40:59.937351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937183 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-kubelet\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:40:59.937351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.937205 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-slash\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:40:59.964632 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.964572 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f87xz" Sep 17 17:40:59.973596 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:40:59.973561 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f87xz" Sep 17 17:41:00.037579 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037488 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-sys\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.037579 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037536 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-lib-modules\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.037579 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037562 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-tuned\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037584 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-cnibin\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037608 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037651 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-socket-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037676 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037698 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-modprobe-d\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037739 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-cni-multus\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037765 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t6fw\" (UniqueName: \"kubernetes.io/projected/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-kube-api-access-9t6fw\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037787 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysconfig\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.037820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037810 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-cni-bin\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037834 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-etc-kubernetes\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037864 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ctbf\" (UniqueName: \"kubernetes.io/projected/86df2dae-c4b8-4042-a75a-29e68a739826-kube-api-access-5ctbf\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037887 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-var-lib-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037909 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-log-socket\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037934 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3520ab4e-6be6-42fb-888f-cbd57c6e7506-konnectivity-ca\") pod \"konnectivity-agent-mz7nt\" (UID: \"3520ab4e-6be6-42fb-888f-cbd57c6e7506\") " pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037959 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-multus-certs\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037974 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-systemd-units\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.037987 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-systemd\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038001 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-cni-bin\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038015 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc4fz\" (UniqueName: \"kubernetes.io/projected/9eedb140-05c7-48ce-8171-13998ef8be9e-kube-api-access-vc4fz\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038030 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d3e728a9-fb95-4223-9920-37a2972a101c-kubelet-config\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038044 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42qnm\" (UniqueName: \"kubernetes.io/projected/7191de83-8ab8-4a34-91ec-2bc5fb95368b-kube-api-access-42qnm\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038060 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-run-netns\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038073 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-ovnkube-config\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038087 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-host\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038102 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whtm9\" (UniqueName: \"kubernetes.io/projected/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-kube-api-access-whtm9\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038115 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-run\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.038161 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038130 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-host\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038143 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038161 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-env-overrides\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038175 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-serviceca\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038193 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7191de83-8ab8-4a34-91ec-2bc5fb95368b-iptables-alerter-script\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038210 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7191de83-8ab8-4a34-91ec-2bc5fb95368b-host-slash\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038224 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d0ec105-91bf-460a-9fcf-307246b66218-hosts-file\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038238 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysctl-d\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038252 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-etc-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038278 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-ovnkube-script-lib\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038295 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038311 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-sys-fs\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038327 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-socket-dir-parent\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038343 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038359 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp9hf\" (UniqueName: \"kubernetes.io/projected/4d0ec105-91bf-460a-9fcf-307246b66218-kube-api-access-jp9hf\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038374 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-device-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038387 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3520ab4e-6be6-42fb-888f-cbd57c6e7506-agent-certs\") pod \"konnectivity-agent-mz7nt\" (UID: \"3520ab4e-6be6-42fb-888f-cbd57c6e7506\") " pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:00.038750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038401 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-tmp\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038415 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2np97\" (UniqueName: \"kubernetes.io/projected/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-kube-api-access-2np97\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038429 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-kubelet\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038445 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d3e728a9-fb95-4223-9920-37a2972a101c-dbus\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038462 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86df2dae-c4b8-4042-a75a-29e68a739826-cni-binary-copy\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038477 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-netns\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038501 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-node-log\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038516 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038533 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038548 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-cni-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038563 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-k8s-cni-cncf-io\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038580 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-conf-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038600 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcwrn\" (UniqueName: \"kubernetes.io/projected/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-kube-api-access-gcwrn\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038639 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-system-cni-dir\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038664 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-registration-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038690 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038714 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysctl-conf\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.039467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038761 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-host\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038767 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-run-netns\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038833 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-cnibin\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038868 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-modprobe-d\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038870 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038892 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-sys\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038892 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysctl-conf\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038926 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7191de83-8ab8-4a34-91ec-2bc5fb95368b-host-slash\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038942 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d0ec105-91bf-460a-9fcf-307246b66218-hosts-file\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.038994 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-system-cni-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039093 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-cni-multus\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039098 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-var-lib-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039148 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-ovn\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039152 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysconfig\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039175 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-run-ovn-kubernetes\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.039190 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039204 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-cnibin\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039213 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-sysctl-d\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039230 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d0ec105-91bf-460a-9fcf-307246b66218-tmp-dir\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:41:00.040273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039250 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-cni-bin\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.039271 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret podName:d3e728a9-fb95-4223-9920-37a2972a101c nodeName:}" failed. No retries permitted until 2026-09-17 17:41:00.539249774 +0000 UTC m=+4.688397175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret") pod "global-pull-secret-syncer-xs9vn" (UID: "d3e728a9-fb95-4223-9920-37a2972a101c") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039289 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-kubelet\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039317 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-slash\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039339 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-socket-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039342 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-cni-netd\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039384 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-cni-netd\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039388 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039417 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/720b09de-1c22-4a55-88f6-66aca41add21-ovn-node-metrics-cert\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039425 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-serviceca\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039453 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th9mr\" (UniqueName: \"kubernetes.io/projected/720b09de-1c22-4a55-88f6-66aca41add21-kube-api-access-th9mr\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039476 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d3e728a9-fb95-4223-9920-37a2972a101c-kubelet-config\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039481 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-systemd\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039502 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-cni-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039513 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-hostroot\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039520 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-systemd\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039541 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.041178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039570 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-kubernetes\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039582 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7191de83-8ab8-4a34-91ec-2bc5fb95368b-iptables-alerter-script\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039597 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-var-lib-kubelet\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039604 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039641 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-ovnkube-script-lib\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039661 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-conf-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039667 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-var-lib-kubelet\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039542 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-run\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039679 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-cni-bin\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039703 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-netns\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039716 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-kubernetes\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039722 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-system-cni-dir\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039737 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-run-ovn-kubernetes\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039748 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-etc-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039788 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-sys-fs\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039793 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-hostroot\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039781 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86df2dae-c4b8-4042-a75a-29e68a739826-multus-daemon-config\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039570 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-k8s-cni-cncf-io\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.042024 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039818 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-lib-modules\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039848 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-node-log\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039886 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-device-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039889 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-var-lib-kubelet\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039932 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-slash\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040035 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-host-run-multus-certs\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040078 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040149 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040207 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-systemd\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040289 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040345 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-systemd-units\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039701 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-ovn\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040417 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040504 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d0ec105-91bf-460a-9fcf-307246b66218-tmp-dir\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040501 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9eedb140-05c7-48ce-8171-13998ef8be9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040291 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040540 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86df2dae-c4b8-4042-a75a-29e68a739826-multus-daemon-config\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040543 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-multus-socket-dir-parent\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.043084 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.040668 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040649 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-env-overrides\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040738 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86df2dae-c4b8-4042-a75a-29e68a739826-etc-kubernetes\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.040763 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs podName:dd1e8497-38f8-4cf2-b979-8cbadd3f4d38 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:00.540743418 +0000 UTC m=+4.689890817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs") pod "network-metrics-daemon-xnrfz" (UID: "dd1e8497-38f8-4cf2-b979-8cbadd3f4d38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.039895 2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040808 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-cnibin\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040879 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d3e728a9-fb95-4223-9920-37a2972a101c-dbus\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040880 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3520ab4e-6be6-42fb-888f-cbd57c6e7506-konnectivity-ca\") pod \"konnectivity-agent-mz7nt\" (UID: \"3520ab4e-6be6-42fb-888f-cbd57c6e7506\") " pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.040935 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-host-kubelet\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.041013 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-run-openvswitch\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.041061 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-registration-dir\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.041090 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/720b09de-1c22-4a55-88f6-66aca41add21-ovnkube-config\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.041239 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9eedb140-05c7-48ce-8171-13998ef8be9e-system-cni-dir\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.041279 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-host\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.041310 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/720b09de-1c22-4a55-88f6-66aca41add21-log-socket\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.041344 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86df2dae-c4b8-4042-a75a-29e68a739826-cni-binary-copy\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.041776 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" event={"ID":"06841d19bfd80202ae04f77c7199dad4","Type":"ContainerStarted","Data":"8445384a42da506527563ad81052828393a3abbb97992db9d51dbd275f4b7d4b"} Sep 17 17:41:00.043917 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.043368 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" event={"ID":"2776e3fe0741747d431c1bf536cbf4b1","Type":"ContainerStarted","Data":"918f4c81fc8acbea99768ed5d6a1857e2c6a3b43328d1057720e93e733669e2d"} Sep 17 17:41:00.044851 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.044567 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-tmp\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.044851 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.044669 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-etc-tuned\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.044851 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.044835 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3520ab4e-6be6-42fb-888f-cbd57c6e7506-agent-certs\") pod \"konnectivity-agent-mz7nt\" (UID: \"3520ab4e-6be6-42fb-888f-cbd57c6e7506\") " pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:00.046086 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.046062 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/720b09de-1c22-4a55-88f6-66aca41add21-ovn-node-metrics-cert\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.049786 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.049576 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42qnm\" (UniqueName: \"kubernetes.io/projected/7191de83-8ab8-4a34-91ec-2bc5fb95368b-kube-api-access-42qnm\") pod \"iptables-alerter-8spf2\" (UID: \"7191de83-8ab8-4a34-91ec-2bc5fb95368b\") " pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:41:00.050020 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.050003 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc4fz\" (UniqueName: \"kubernetes.io/projected/9eedb140-05c7-48ce-8171-13998ef8be9e-kube-api-access-vc4fz\") pod \"multus-additional-cni-plugins-pcvrb\" (UID: \"9eedb140-05c7-48ce-8171-13998ef8be9e\") " pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.051055 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.050558 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:41:00.051055 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.050583 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:41:00.051055 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.050598 2586 projected.go:196] Error preparing data for projected volume kube-api-access-2nn9j for pod openshift-network-diagnostics/network-check-target-wf9fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:00.051055 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.050702 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j podName:de127252-05c1-43c0-aba1-095c828851f9 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:00.550686611 +0000 UTC m=+4.699834009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2nn9j" (UniqueName: "kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j") pod "network-check-target-wf9fj" (UID: "de127252-05c1-43c0-aba1-095c828851f9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:00.053204 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.053178 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp9hf\" (UniqueName: \"kubernetes.io/projected/4d0ec105-91bf-460a-9fcf-307246b66218-kube-api-access-jp9hf\") pod \"node-resolver-kz957\" (UID: \"4d0ec105-91bf-460a-9fcf-307246b66218\") " pod="openshift-dns/node-resolver-kz957" Sep 17 17:41:00.053560 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.053540 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whtm9\" (UniqueName: \"kubernetes.io/projected/6d3c9ef5-2029-476f-bc07-5cc54bc5ad18-kube-api-access-whtm9\") pod \"node-ca-tnn9m\" (UID: \"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18\") " pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:41:00.056506 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.056478 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcwrn\" (UniqueName: \"kubernetes.io/projected/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-kube-api-access-gcwrn\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:00.057696 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.056986 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th9mr\" (UniqueName: \"kubernetes.io/projected/720b09de-1c22-4a55-88f6-66aca41add21-kube-api-access-th9mr\") pod \"ovnkube-node-6mvdd\" (UID: \"720b09de-1c22-4a55-88f6-66aca41add21\") " pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.057696 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.057445 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t6fw\" (UniqueName: \"kubernetes.io/projected/fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c-kube-api-access-9t6fw\") pod \"aws-ebs-csi-driver-node-lb7t4\" (UID: \"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.058050 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.057947 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2np97\" (UniqueName: \"kubernetes.io/projected/9b7d1d57-83de-4fbe-9816-3ac19ea140ea-kube-api-access-2np97\") pod \"tuned-ndfck\" (UID: \"9b7d1d57-83de-4fbe-9816-3ac19ea140ea\") " pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.058235 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.058211 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ctbf\" (UniqueName: \"kubernetes.io/projected/86df2dae-c4b8-4042-a75a-29e68a739826-kube-api-access-5ctbf\") pod \"multus-v56s5\" (UID: \"86df2dae-c4b8-4042-a75a-29e68a739826\") " pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.193946 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.193914 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:00.203685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.203652 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:00.214681 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.214645 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" Sep 17 17:41:00.219392 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.219365 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8spf2" Sep 17 17:41:00.227155 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.227127 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kz957" Sep 17 17:41:00.234952 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.234923 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ndfck" Sep 17 17:41:00.241902 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.241879 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tnn9m" Sep 17 17:41:00.249516 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.249492 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" Sep 17 17:41:00.256388 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.256370 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v56s5" Sep 17 17:41:00.398228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.398192 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 17:41:00.544699 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.544663 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:00.544856 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.544710 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:00.544856 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.544829 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:00.544856 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.544832 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:00.544951 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.544884 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret podName:d3e728a9-fb95-4223-9920-37a2972a101c nodeName:}" failed. No retries permitted until 2026-09-17 17:41:01.544871399 +0000 UTC m=+5.694018797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret") pod "global-pull-secret-syncer-xs9vn" (UID: "d3e728a9-fb95-4223-9920-37a2972a101c") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:00.544951 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.544898 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs podName:dd1e8497-38f8-4cf2-b979-8cbadd3f4d38 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:01.544891663 +0000 UTC m=+5.694039061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs") pod "network-metrics-daemon-xnrfz" (UID: "dd1e8497-38f8-4cf2-b979-8cbadd3f4d38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:00.646102 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.646064 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:00.646299 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.646276 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:41:00.646346 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.646309 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:41:00.646346 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.646322 2586 projected.go:196] Error preparing data for projected volume kube-api-access-2nn9j for pod openshift-network-diagnostics/network-check-target-wf9fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:00.646421 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:00.646390 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j podName:de127252-05c1-43c0-aba1-095c828851f9 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:01.646370471 +0000 UTC m=+5.795517873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2nn9j" (UniqueName: "kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j") pod "network-check-target-wf9fj" (UID: "de127252-05c1-43c0-aba1-095c828851f9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:00.964414 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:00.964391 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eedb140_05c7_48ce_8171_13998ef8be9e.slice/crio-d5eb73cf4b4bb8f8fc9ac830b37c5d57d90a56fc903b2f2e9188d7b1b165e966 WatchSource:0}: Error finding container d5eb73cf4b4bb8f8fc9ac830b37c5d57d90a56fc903b2f2e9188d7b1b165e966: Status 404 returned error can't find the container with id d5eb73cf4b4bb8f8fc9ac830b37c5d57d90a56fc903b2f2e9188d7b1b165e966 Sep 17 17:41:00.971268 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:00.971233 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b7d1d57_83de_4fbe_9816_3ac19ea140ea.slice/crio-9cd2bcccc335a1d764e1f1539e9a7f59c8fd94e5cf2a3bc1dcc7ed470bc05f87 WatchSource:0}: Error finding container 9cd2bcccc335a1d764e1f1539e9a7f59c8fd94e5cf2a3bc1dcc7ed470bc05f87: Status 404 returned error can't find the container with id 9cd2bcccc335a1d764e1f1539e9a7f59c8fd94e5cf2a3bc1dcc7ed470bc05f87 Sep 17 17:41:00.973356 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:00.973329 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3520ab4e_6be6_42fb_888f_cbd57c6e7506.slice/crio-ab506fad304253b50266b8754b29032f2b4b3698ad98b99a7fc314b9ac6828c1 WatchSource:0}: Error finding container ab506fad304253b50266b8754b29032f2b4b3698ad98b99a7fc314b9ac6828c1: Status 404 returned error can't find the container with id ab506fad304253b50266b8754b29032f2b4b3698ad98b99a7fc314b9ac6828c1 Sep 17 17:41:00.974808 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:00.974787 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3c18ef_d3cd_480e_9ea5_3328db7b2f9c.slice/crio-92b81e17b92bac08fdcbeb075ecc73f90c75cd53d9a00f7b0112b89c396c23d4 WatchSource:0}: Error finding container 92b81e17b92bac08fdcbeb075ecc73f90c75cd53d9a00f7b0112b89c396c23d4: Status 404 returned error can't find the container with id 92b81e17b92bac08fdcbeb075ecc73f90c75cd53d9a00f7b0112b89c396c23d4 Sep 17 17:41:00.975681 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:00.975598 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720b09de_1c22_4a55_88f6_66aca41add21.slice/crio-2a2776f14218597188f91f9787889b95c588029a1f9ad37c0f21eb708cd0f04e WatchSource:0}: Error finding container 2a2776f14218597188f91f9787889b95c588029a1f9ad37c0f21eb708cd0f04e: Status 404 returned error can't find the container with id 2a2776f14218597188f91f9787889b95c588029a1f9ad37c0f21eb708cd0f04e Sep 17 17:41:00.976162 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.976122 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:35:59 +0000 UTC" deadline="2028-03-13 06:07:48.892090961 +0000 UTC" Sep 17 17:41:00.976162 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:00.976151 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13020h26m47.91594382s" Sep 17 17:41:00.978030 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:00.977934 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0ec105_91bf_460a_9fcf_307246b66218.slice/crio-51bca93300b22b3cf6636e239622777956c35341883b314c74176dd35343a550 WatchSource:0}: Error finding container 51bca93300b22b3cf6636e239622777956c35341883b314c74176dd35343a550: Status 404 returned error can't find the container with id 51bca93300b22b3cf6636e239622777956c35341883b314c74176dd35343a550 Sep 17 17:41:00.979447 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:00.979419 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86df2dae_c4b8_4042_a75a_29e68a739826.slice/crio-af160d18db44df550dac066caf37f08885cd9f08de4086e87c91bcc484257875 WatchSource:0}: Error finding container af160d18db44df550dac066caf37f08885cd9f08de4086e87c91bcc484257875: Status 404 returned error can't find the container with id af160d18db44df550dac066caf37f08885cd9f08de4086e87c91bcc484257875 Sep 17 17:41:01.033546 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.033402 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:01.033666 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.033645 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:01.046810 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.046783 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"2a2776f14218597188f91f9787889b95c588029a1f9ad37c0f21eb708cd0f04e"} Sep 17 17:41:01.048078 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.048047 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tnn9m" event={"ID":"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18","Type":"ContainerStarted","Data":"89796672e825f042ceee70ab43baae9a8e55851588e496693517ac187c663316"} Sep 17 17:41:01.049273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.049247 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" event={"ID":"9eedb140-05c7-48ce-8171-13998ef8be9e","Type":"ContainerStarted","Data":"d5eb73cf4b4bb8f8fc9ac830b37c5d57d90a56fc903b2f2e9188d7b1b165e966"} Sep 17 17:41:01.050908 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.050877 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mz7nt" event={"ID":"3520ab4e-6be6-42fb-888f-cbd57c6e7506","Type":"ContainerStarted","Data":"ab506fad304253b50266b8754b29032f2b4b3698ad98b99a7fc314b9ac6828c1"} Sep 17 17:41:01.052104 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.052075 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" event={"ID":"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c","Type":"ContainerStarted","Data":"92b81e17b92bac08fdcbeb075ecc73f90c75cd53d9a00f7b0112b89c396c23d4"} Sep 17 17:41:01.053252 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.053229 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kz957" event={"ID":"4d0ec105-91bf-460a-9fcf-307246b66218","Type":"ContainerStarted","Data":"51bca93300b22b3cf6636e239622777956c35341883b314c74176dd35343a550"} Sep 17 17:41:01.054446 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.054419 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v56s5" event={"ID":"86df2dae-c4b8-4042-a75a-29e68a739826","Type":"ContainerStarted","Data":"af160d18db44df550dac066caf37f08885cd9f08de4086e87c91bcc484257875"} Sep 17 17:41:01.055670 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.055644 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ndfck" event={"ID":"9b7d1d57-83de-4fbe-9816-3ac19ea140ea","Type":"ContainerStarted","Data":"9cd2bcccc335a1d764e1f1539e9a7f59c8fd94e5cf2a3bc1dcc7ed470bc05f87"} Sep 17 17:41:01.056814 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.056791 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8spf2" event={"ID":"7191de83-8ab8-4a34-91ec-2bc5fb95368b","Type":"ContainerStarted","Data":"570d4ff1fbc94ec1c2bb80687df4b9a75452a20d3152af7678d9cc40a5db1349"} Sep 17 17:41:01.553847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.553046 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:01.553847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.553100 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:01.553847 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.553252 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:01.553847 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.553313 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret podName:d3e728a9-fb95-4223-9920-37a2972a101c nodeName:}" failed. No retries permitted until 2026-09-17 17:41:03.553295289 +0000 UTC m=+7.702442692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret") pod "global-pull-secret-syncer-xs9vn" (UID: "d3e728a9-fb95-4223-9920-37a2972a101c") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:01.553847 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.553711 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:01.553847 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.553767 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs podName:dd1e8497-38f8-4cf2-b979-8cbadd3f4d38 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:03.553747335 +0000 UTC m=+7.702894738 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs") pod "network-metrics-daemon-xnrfz" (UID: "dd1e8497-38f8-4cf2-b979-8cbadd3f4d38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:01.655017 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.654408 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:01.655017 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.654644 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:41:01.655017 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.654663 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:41:01.655017 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.654676 2586 projected.go:196] Error preparing data for projected volume kube-api-access-2nn9j for pod openshift-network-diagnostics/network-check-target-wf9fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:01.655017 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:01.654733 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j podName:de127252-05c1-43c0-aba1-095c828851f9 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:03.65471543 +0000 UTC m=+7.803862834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2nn9j" (UniqueName: "kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j") pod "network-check-target-wf9fj" (UID: "de127252-05c1-43c0-aba1-095c828851f9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:01.977232 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.977155 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:35:59 +0000 UTC" deadline="2028-04-14 20:35:29.245575297 +0000 UTC" Sep 17 17:41:01.977232 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:01.977194 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13802h54m27.268384943s" Sep 17 17:41:02.040080 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:02.040039 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:02.040261 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:02.040172 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:02.040591 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:02.040562 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:02.040724 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:02.040682 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:02.078793 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:02.078755 2586 generic.go:356] "Generic (PLEG): container finished" podID="2776e3fe0741747d431c1bf536cbf4b1" containerID="5ae075cc47a6b0212c4b3f389374cc2d867a94d68b4a0095b1bbd3ddd861e771" exitCode=0 Sep 17 17:41:02.079088 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:02.079052 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" event={"ID":"2776e3fe0741747d431c1bf536cbf4b1","Type":"ContainerDied","Data":"5ae075cc47a6b0212c4b3f389374cc2d867a94d68b4a0095b1bbd3ddd861e771"} Sep 17 17:41:02.088934 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:02.088896 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" event={"ID":"06841d19bfd80202ae04f77c7199dad4","Type":"ContainerStarted","Data":"1587da816cc88bfb868306a394b70d21b7720958d29c94226f5817c1a9f65055"} Sep 17 17:41:02.117591 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:02.117512 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-30.ec2.internal" podStartSLOduration=3.11748957 podStartE2EDuration="3.11748957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:02.117233687 +0000 UTC m=+6.266381108" watchObservedRunningTime="2026-09-17 17:41:02.11748957 +0000 UTC m=+6.266636992" Sep 17 17:41:03.032478 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:03.032435 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:03.033009 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.032609 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:03.120417 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:03.120285 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" event={"ID":"2776e3fe0741747d431c1bf536cbf4b1","Type":"ContainerStarted","Data":"3cf711c2b07b52ba1194e98ead6bb26038ac8697ec083d4e331fba4f57f04da0"} Sep 17 17:41:03.570666 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:03.569888 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:03.570666 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:03.569947 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:03.570666 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.570090 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:03.570666 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.570152 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret podName:d3e728a9-fb95-4223-9920-37a2972a101c nodeName:}" failed. No retries permitted until 2026-09-17 17:41:07.570134362 +0000 UTC m=+11.719281779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret") pod "global-pull-secret-syncer-xs9vn" (UID: "d3e728a9-fb95-4223-9920-37a2972a101c") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:03.570666 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.570218 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:03.570666 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.570250 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs podName:dd1e8497-38f8-4cf2-b979-8cbadd3f4d38 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:07.570239934 +0000 UTC m=+11.719387331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs") pod "network-metrics-daemon-xnrfz" (UID: "dd1e8497-38f8-4cf2-b979-8cbadd3f4d38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:03.670683 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:03.670525 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:03.671488 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.671032 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:41:03.671488 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.671056 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:41:03.671488 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.671069 2586 projected.go:196] Error preparing data for projected volume kube-api-access-2nn9j for pod openshift-network-diagnostics/network-check-target-wf9fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:03.671488 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:03.671130 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j podName:de127252-05c1-43c0-aba1-095c828851f9 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:07.671111569 +0000 UTC m=+11.820258971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2nn9j" (UniqueName: "kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j") pod "network-check-target-wf9fj" (UID: "de127252-05c1-43c0-aba1-095c828851f9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:04.033415 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:04.033106 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:04.033415 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:04.033240 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:04.034045 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:04.033580 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:04.034045 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:04.033664 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:05.032641 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:05.032578 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:05.032827 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:05.032747 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:06.037278 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:06.036591 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:06.037278 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:06.036764 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:06.037278 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:06.037136 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:06.037278 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:06.037227 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:07.033456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:07.033416 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:07.033660 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.033542 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:07.603854 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:07.603815 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:07.604370 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:07.603912 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:07.604370 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.604031 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:07.604370 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.604088 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs podName:dd1e8497-38f8-4cf2-b979-8cbadd3f4d38 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:15.604069594 +0000 UTC m=+19.753217007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs") pod "network-metrics-daemon-xnrfz" (UID: "dd1e8497-38f8-4cf2-b979-8cbadd3f4d38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:07.604515 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.604483 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:07.604571 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.604530 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret podName:d3e728a9-fb95-4223-9920-37a2972a101c nodeName:}" failed. No retries permitted until 2026-09-17 17:41:15.604515537 +0000 UTC m=+19.753662950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret") pod "global-pull-secret-syncer-xs9vn" (UID: "d3e728a9-fb95-4223-9920-37a2972a101c") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:07.704535 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:07.704498 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:07.704745 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.704717 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:41:07.704745 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.704744 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:41:07.704859 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.704759 2586 projected.go:196] Error preparing data for projected volume kube-api-access-2nn9j for pod openshift-network-diagnostics/network-check-target-wf9fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:07.704859 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:07.704817 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j podName:de127252-05c1-43c0-aba1-095c828851f9 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:15.704798917 +0000 UTC m=+19.853946320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2nn9j" (UniqueName: "kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j") pod "network-check-target-wf9fj" (UID: "de127252-05c1-43c0-aba1-095c828851f9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:08.036467 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:08.036396 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:08.036639 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:08.036504 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:08.036639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:08.036563 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:08.036744 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:08.036679 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:09.033293 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:09.033257 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:09.033847 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:09.033410 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:10.032906 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:10.032869 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:10.033089 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:10.033002 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:10.033458 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:10.033435 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:10.033843 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:10.033544 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:11.033169 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:11.033133 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:11.033352 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:11.033263 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:12.034651 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:12.032986 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:12.034651 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:12.032987 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:12.034651 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:12.033121 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:12.034651 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:12.033228 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:13.032970 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:13.032928 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:13.033154 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:13.033076 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:14.033242 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:14.033201 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:14.033732 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:14.033250 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:14.033732 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:14.033335 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:14.033732 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:14.033455 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:15.033264 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:15.033234 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:15.033722 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.033347 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:15.662533 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:15.662492 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:15.662706 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:15.662542 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:15.662706 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.662687 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:15.662793 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.662751 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret podName:d3e728a9-fb95-4223-9920-37a2972a101c nodeName:}" failed. No retries permitted until 2026-09-17 17:41:31.66273827 +0000 UTC m=+35.811885667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret") pod "global-pull-secret-syncer-xs9vn" (UID: "d3e728a9-fb95-4223-9920-37a2972a101c") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:15.663100 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.663086 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:15.663130 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.663125 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs podName:dd1e8497-38f8-4cf2-b979-8cbadd3f4d38 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:31.663113969 +0000 UTC m=+35.812261367 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs") pod "network-metrics-daemon-xnrfz" (UID: "dd1e8497-38f8-4cf2-b979-8cbadd3f4d38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:15.763948 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:15.763907 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:15.764133 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.764114 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:41:15.764204 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.764145 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:41:15.764204 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.764160 2586 projected.go:196] Error preparing data for projected volume kube-api-access-2nn9j for pod openshift-network-diagnostics/network-check-target-wf9fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:15.764291 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:15.764273 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j podName:de127252-05c1-43c0-aba1-095c828851f9 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:31.764255097 +0000 UTC m=+35.913402514 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2nn9j" (UniqueName: "kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j") pod "network-check-target-wf9fj" (UID: "de127252-05c1-43c0-aba1-095c828851f9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:16.032711 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:16.032606 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:16.032711 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:16.032673 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:16.032915 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:16.032773 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:16.033214 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:16.033188 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:17.032716 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:17.032671 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:17.033199 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:17.032822 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:18.033918 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:18.033872 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:18.034357 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:18.034046 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:18.034411 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:18.034357 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:18.034457 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:18.034429 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:19.033113 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.032874 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:19.033312 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:19.033260 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:19.164008 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.163969 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v56s5" event={"ID":"86df2dae-c4b8-4042-a75a-29e68a739826","Type":"ContainerStarted","Data":"41619cbd1ebec3388f7b5263ce5cba762bc5b2b27a3cb1a6a712fe6d7371fe6c"} Sep 17 17:41:19.166405 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.166371 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ndfck" event={"ID":"9b7d1d57-83de-4fbe-9816-3ac19ea140ea","Type":"ContainerStarted","Data":"808f7032e7b55f30911e7a35db76c47e14e25471c46c693f4dc9804b29d0014c"} Sep 17 17:41:19.170365 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.170328 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"edf735ffdfe5455ce31d5a481598a67f1e5382cc00cc3249d9f36c1cb32f33fb"} Sep 17 17:41:19.171735 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.171710 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tnn9m" event={"ID":"6d3c9ef5-2029-476f-bc07-5cc54bc5ad18","Type":"ContainerStarted","Data":"cfa7bf0b3da221c0debfad4eff4799fcb06c4fd8b3154362e2123b34604ff996"} Sep 17 17:41:19.173893 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.173867 2586 generic.go:356] "Generic (PLEG): container finished" podID="9eedb140-05c7-48ce-8171-13998ef8be9e" containerID="f1a4f03b54e01926c34638cc63b5a16497aedec20a5120c5ba8a410c9940fc96" exitCode=0 Sep 17 17:41:19.174011 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.173965 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" event={"ID":"9eedb140-05c7-48ce-8171-13998ef8be9e","Type":"ContainerDied","Data":"f1a4f03b54e01926c34638cc63b5a16497aedec20a5120c5ba8a410c9940fc96"} Sep 17 17:41:19.175798 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.175749 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mz7nt" event={"ID":"3520ab4e-6be6-42fb-888f-cbd57c6e7506","Type":"ContainerStarted","Data":"313b7148a24125218bb8b34ec22c7081289b4f45d46f1abdd3ec029b41e9d381"} Sep 17 17:41:19.177410 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.177385 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" event={"ID":"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c","Type":"ContainerStarted","Data":"fa003a2c2a2dc8eb3fb281b1f0b8aeea7378f08ad304c94690dc545e70939706"} Sep 17 17:41:19.178991 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.178968 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kz957" event={"ID":"4d0ec105-91bf-460a-9fcf-307246b66218","Type":"ContainerStarted","Data":"98ad32eb552fb2cc3f5b4e62a86b687556f7e912de589135a1bc3116ed229c2a"} Sep 17 17:41:19.183225 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.183176 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-30.ec2.internal" podStartSLOduration=20.183162585 podStartE2EDuration="20.183162585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:03.136966952 +0000 UTC m=+7.286114373" watchObservedRunningTime="2026-09-17 17:41:19.183162585 +0000 UTC m=+23.332310006" Sep 17 17:41:19.183333 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.183247 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v56s5" podStartSLOduration=3.429219276 podStartE2EDuration="21.183243175s" totalImagesPullingTime="17.754023899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:19.182731824 +0000 UTC m=+23.331879245" watchObservedRunningTime="2026-09-17 17:41:19.183243175 +0000 UTC m=+23.332390592" Sep 17 17:41:19.219308 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.219131 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mz7nt" podStartSLOduration=3.480869321 podStartE2EDuration="21.219114429s" totalImagesPullingTime="17.738245108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:19.197317206 +0000 UTC m=+23.346464627" watchObservedRunningTime="2026-09-17 17:41:19.219114429 +0000 UTC m=+23.368261849" Sep 17 17:41:19.235973 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.235922 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kz957" podStartSLOduration=3.50482884 podStartE2EDuration="21.235909419s" totalImagesPullingTime="17.731080579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:19.235904639 +0000 UTC m=+23.385052059" watchObservedRunningTime="2026-09-17 17:41:19.235909419 +0000 UTC m=+23.385056851" Sep 17 17:41:19.296231 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.296184 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ndfck" podStartSLOduration=3.556309708 podStartE2EDuration="21.296139946s" totalImagesPullingTime="17.739830238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:19.267356698 +0000 UTC m=+23.416504118" watchObservedRunningTime="2026-09-17 17:41:19.296139946 +0000 UTC m=+23.445287366" Sep 17 17:41:19.296397 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.296370 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tnn9m" podStartSLOduration=3.550036821 podStartE2EDuration="21.296358008s" totalImagesPullingTime="17.746321187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:19.296074566 +0000 UTC m=+23.445221988" watchObservedRunningTime="2026-09-17 17:41:19.296358008 +0000 UTC m=+23.445505429" Sep 17 17:41:19.827843 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:19.827812 2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 17:41:20.004495 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.004267 2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T17:41:19.827841444Z","UUID":"35dd5428-2308-4960-a174-20ea0cacc30f","Handler":null,"Name":"","Endpoint":""} Sep 17 17:41:20.005871 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.005850 2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 17:41:20.005989 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.005884 2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 17:41:20.033481 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.033446 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:20.033692 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.033453 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:20.033692 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:20.033575 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:20.033819 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:20.033682 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:20.187319 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.187269 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"53cf7738a264629af58a78f45cc4c04aa56dd32f9681e3b41f07eaa5160b3163"} Sep 17 17:41:20.187867 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.187360 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"6098c2881e6c4d3fc17f64442f6356594413549f05ddef7d244f9462a1b39410"} Sep 17 17:41:20.187867 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.187399 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"98963f34f3f1e2088e8cae6c6769bcc6ca10cd38887b293aa380e5364334c585"} Sep 17 17:41:20.187867 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.187438 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"1e874c083ddb5667303435fa247323f4af227fc22034c4ed85c4023980996a7f"} Sep 17 17:41:20.187867 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.187481 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"69a0646fc15d92019ab6ba2cb4709fb3112fd17860d0e87fc2685f84eabf02e2"} Sep 17 17:41:20.189842 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:20.189810 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" event={"ID":"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c","Type":"ContainerStarted","Data":"16d0ae7f9837f04a370c45fd4e6bfb5fe2a9b45d9161462e1324a0c83616dab5"} Sep 17 17:41:21.033432 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:21.033328 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:21.033601 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:21.033513 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:21.195040 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:21.195001 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" event={"ID":"fd3c18ef-d3cd-480e-9ea5-3328db7b2f9c","Type":"ContainerStarted","Data":"bf547431364321784f54f268728a8c4013d60d01196a2422f547e416b6858dea"} Sep 17 17:41:21.196692 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:21.196668 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8spf2" event={"ID":"7191de83-8ab8-4a34-91ec-2bc5fb95368b","Type":"ContainerStarted","Data":"d9491e50ca078577f491cabd772a206265607abafaacc24984547d89db6e5c92"} Sep 17 17:41:21.216758 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:21.216703 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lb7t4" podStartSLOduration=3.895932065 podStartE2EDuration="23.216679628s" totalImagesPullingTime="19.320747563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:21.216501153 +0000 UTC m=+25.365648573" watchObservedRunningTime="2026-09-17 17:41:21.216679628 +0000 UTC m=+25.365827048" Sep 17 17:41:21.233722 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:21.233657 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8spf2" podStartSLOduration=5.482841232 podStartE2EDuration="23.233635434s" totalImagesPullingTime="17.750794202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:21.232717752 +0000 UTC m=+25.381865173" watchObservedRunningTime="2026-09-17 17:41:21.233635434 +0000 UTC m=+25.382782853" Sep 17 17:41:22.032598 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:22.032559 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:22.032806 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:22.032560 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:22.032806 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:22.032719 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:22.032927 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:22.032840 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:22.203391 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:22.203350 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"b8a19b0749637216fa08032b67f978005af1dd7e62c8d0823d43042fee365e56"} Sep 17 17:41:23.033448 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:23.033411 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:23.033670 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:23.033540 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:23.237434 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:23.237396 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:23.238481 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:23.238299 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:24.033594 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.033352 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:24.033776 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.033405 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:24.033776 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:24.033725 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:24.033840 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:24.033824 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:24.212571 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.212472 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" event={"ID":"720b09de-1c22-4a55-88f6-66aca41add21","Type":"ContainerStarted","Data":"ef5069c5c7a86e78f6057141d43a9dafcd7f81e162ddc3b504610ac8d252a842"} Sep 17 17:41:24.212854 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.212831 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:24.214458 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.214434 2586 generic.go:356] "Generic (PLEG): container finished" podID="9eedb140-05c7-48ce-8171-13998ef8be9e" containerID="6b65c44a8c77e2f65f3aed20408154b3c6e9726dceb1365c386f987874d0ee7b" exitCode=0 Sep 17 17:41:24.214585 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.214528 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" event={"ID":"9eedb140-05c7-48ce-8171-13998ef8be9e","Type":"ContainerDied","Data":"6b65c44a8c77e2f65f3aed20408154b3c6e9726dceb1365c386f987874d0ee7b"} Sep 17 17:41:24.214793 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.214776 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:24.215372 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.215303 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mz7nt" Sep 17 17:41:24.229411 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.229385 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:24.249278 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:24.248929 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" podStartSLOduration=8.280000792 podStartE2EDuration="26.248908963s" totalImagesPullingTime="17.968908171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:24.247949704 +0000 UTC m=+28.397097124" watchObservedRunningTime="2026-09-17 17:41:24.248908963 +0000 UTC m=+28.398056384" Sep 17 17:41:25.032908 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.032866 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:25.033093 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:25.033022 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:25.217198 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.217171 2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 17 17:41:25.217678 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.217655 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:25.233271 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.233243 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:25.992749 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.991659 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wf9fj"] Sep 17 17:41:25.992749 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.991999 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xnrfz"] Sep 17 17:41:25.992749 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.992044 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xs9vn"] Sep 17 17:41:25.992749 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.992151 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:25.992749 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:25.992302 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:25.992749 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.992401 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:25.992749 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:25.992507 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:25.992749 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:25.992533 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:25.993410 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:25.992759 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:26.221650 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:26.221594 2586 generic.go:356] "Generic (PLEG): container finished" podID="9eedb140-05c7-48ce-8171-13998ef8be9e" containerID="70a119a21e4e904241327aed50767aeb31836c14700397fb077b0047b2507d1d" exitCode=0 Sep 17 17:41:26.221650 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:26.221648 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" event={"ID":"9eedb140-05c7-48ce-8171-13998ef8be9e","Type":"ContainerDied","Data":"70a119a21e4e904241327aed50767aeb31836c14700397fb077b0047b2507d1d"} Sep 17 17:41:26.221870 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:26.221829 2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 17 17:41:27.032871 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:27.032828 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:27.033271 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:27.032942 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:27.223873 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:27.223842 2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 17 17:41:28.036428 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:28.034310 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:28.036428 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:28.034469 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:28.036428 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:28.034964 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:28.036428 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:28.035104 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:28.228449 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:28.228406 2586 generic.go:356] "Generic (PLEG): container finished" podID="9eedb140-05c7-48ce-8171-13998ef8be9e" containerID="dfce433728e71ba0ee5fbe4b584c0952f4b7adc3c7817b32c7912929f47a5020" exitCode=0 Sep 17 17:41:28.228449 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:28.228445 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" event={"ID":"9eedb140-05c7-48ce-8171-13998ef8be9e","Type":"ContainerDied","Data":"dfce433728e71ba0ee5fbe4b584c0952f4b7adc3c7817b32c7912929f47a5020"} Sep 17 17:41:29.032382 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:29.032349 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:29.032631 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:29.032479 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:29.209059 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:29.209006 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:29.209452 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:29.209288 2586 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 17 17:41:29.226649 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:29.226580 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" podUID="720b09de-1c22-4a55-88f6-66aca41add21" containerName="ovnkube-controller" probeResult="failure" output="" Sep 17 17:41:29.239064 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:29.239018 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" podUID="720b09de-1c22-4a55-88f6-66aca41add21" containerName="ovnkube-controller" probeResult="failure" output="" Sep 17 17:41:30.032478 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:30.032435 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:30.032478 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:30.032467 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:30.032772 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:30.032640 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wf9fj" podUID="de127252-05c1-43c0-aba1-095c828851f9" Sep 17 17:41:30.032772 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:30.032720 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xnrfz" podUID="dd1e8497-38f8-4cf2-b979-8cbadd3f4d38" Sep 17 17:41:31.033125 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.033088 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:31.033705 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.033224 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xs9vn" podUID="d3e728a9-fb95-4223-9920-37a2972a101c" Sep 17 17:41:31.687331 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.687287 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:31.687524 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.687349 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:31.687524 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.687459 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:31.687524 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.687498 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:31.687680 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.687536 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs podName:dd1e8497-38f8-4cf2-b979-8cbadd3f4d38 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.687518576 +0000 UTC m=+67.836665997 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs") pod "network-metrics-daemon-xnrfz" (UID: "dd1e8497-38f8-4cf2-b979-8cbadd3f4d38") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 17:41:31.687680 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.687552 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret podName:d3e728a9-fb95-4223-9920-37a2972a101c nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.687545759 +0000 UTC m=+67.836693156 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret") pod "global-pull-secret-syncer-xs9vn" (UID: "d3e728a9-fb95-4223-9920-37a2972a101c") : object "kube-system"/"original-pull-secret" not registered Sep 17 17:41:31.788001 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.787958 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:31.788194 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.788139 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 17:41:31.788194 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.788164 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 17:41:31.788194 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.788174 2586 projected.go:196] Error preparing data for projected volume kube-api-access-2nn9j for pod openshift-network-diagnostics/network-check-target-wf9fj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:31.788307 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:31.788230 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j podName:de127252-05c1-43c0-aba1-095c828851f9 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.788216906 +0000 UTC m=+67.937364304 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2nn9j" (UniqueName: "kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j") pod "network-check-target-wf9fj" (UID: "de127252-05c1-43c0-aba1-095c828851f9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 17:41:31.816171 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.816132 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-30.ec2.internal" event="NodeReady" Sep 17 17:41:31.816351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.816278 2586 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 17:41:31.855701 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.855664 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b8c4ff68f-n4x6c"] Sep 17 17:41:31.887786 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.887725 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9"] Sep 17 17:41:31.887977 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.887864 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.895401 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.895367 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 17 17:41:31.898567 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.896704 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 17 17:41:31.898770 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.898678 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mtf7c\"" Sep 17 17:41:31.899101 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.898897 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 17 17:41:31.901506 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.901481 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 17 17:41:31.901647 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.901595 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-lzhcp"] Sep 17 17:41:31.901798 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.901779 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:31.908888 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.907434 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4hdqv\"" Sep 17 17:41:31.908888 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.907885 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 17:41:31.908888 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.908307 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 17 17:41:31.909130 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.909109 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 17 17:41:31.914515 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.914486 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 17:41:31.927126 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.926183 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-wb4t7"] Sep 17 17:41:31.927126 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.926317 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:31.928677 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.928655 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 17:41:31.928813 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.928776 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 17:41:31.928876 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.928824 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 17 17:41:31.929012 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.928993 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-klhcq\"" Sep 17 17:41:31.929133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.928995 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 17 17:41:31.934418 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.934393 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 17 17:41:31.947994 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.947918 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb"] Sep 17 17:41:31.948137 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.948052 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:31.951707 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.951681 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 17 17:41:31.951858 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.951694 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 17 17:41:31.951995 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.951966 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qq78x\"" Sep 17 17:41:31.951995 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.951965 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:41:31.952242 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.952223 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 17 17:41:31.961792 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.961761 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 17 17:41:31.971579 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.971539 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7b6cb8c868-qgggf"] Sep 17 17:41:31.971761 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.971706 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:31.975677 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.975653 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 17 17:41:31.975890 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.975874 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7cfpq\"" Sep 17 17:41:31.976102 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.976084 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 17 17:41:31.987865 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.987840 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22"] Sep 17 17:41:31.988003 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.987980 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:31.988733 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.988715 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:41:31.989267 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989243 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.989361 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989275 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-installation-pull-secrets\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.989361 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989293 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/97f023e7-bed4-4556-8279-20e645880b88-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:31.989361 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989323 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d22ec60-8779-4a14-90f2-e8abff8d6b61-ca-trust-extracted\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.989361 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989342 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-image-registry-private-configuration\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.989361 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989360 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-trusted-ca\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.989555 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989381 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19467e99-0315-4947-9934-80d423a3e33d-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:31.989555 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989417 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/19467e99-0315-4947-9934-80d423a3e33d-snapshots\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:31.989555 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989437 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmlnj\" (UniqueName: \"kubernetes.io/projected/19467e99-0315-4947-9934-80d423a3e33d-kube-api-access-pmlnj\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:31.989555 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989455 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-758rz\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-kube-api-access-758rz\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.989555 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989473 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19467e99-0315-4947-9934-80d423a3e33d-tmp\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:31.989555 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989521 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19467e99-0315-4947-9934-80d423a3e33d-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:31.989799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989586 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:31.989799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989603 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt5qn\" (UniqueName: \"kubernetes.io/projected/97f023e7-bed4-4556-8279-20e645880b88-kube-api-access-gt5qn\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:31.989799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989637 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19467e99-0315-4947-9934-80d423a3e33d-serving-cert\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:31.989799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989664 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-certificates\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.989799 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.989680 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-bound-sa-token\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:31.991280 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.991261 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 17 17:41:31.991582 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.991560 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 17 17:41:31.991717 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.991701 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 17 17:41:31.991788 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.991779 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 17 17:41:31.992018 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.991999 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 17 17:41:31.992314 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.992295 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 17 17:41:31.992796 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:31.992776 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p96kh\"" Sep 17 17:41:32.011125 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.011088 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b8c4ff68f-n4x6c"] Sep 17 17:41:32.011125 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.011133 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9"] Sep 17 17:41:32.011375 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.011150 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sshtr"] Sep 17 17:41:32.011375 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.011189 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22" Sep 17 17:41:32.015031 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.015009 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mn4dr\"" Sep 17 17:41:32.015503 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.015482 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:41:32.015597 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.015518 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 17 17:41:32.038872 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.038835 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc"] Sep 17 17:41:32.039298 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.038983 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.041952 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.041927 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fglck\"" Sep 17 17:41:32.042260 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.042245 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 17:41:32.042260 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.042256 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 17:41:32.066038 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.066011 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6"] Sep 17 17:41:32.066251 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.066138 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.068833 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.068811 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 17 17:41:32.069062 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.068833 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:41:32.069143 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.068857 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-trcv9\"" Sep 17 17:41:32.069143 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.068870 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 17 17:41:32.069241 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.068889 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 17 17:41:32.090542 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090508 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt5qn\" (UniqueName: \"kubernetes.io/projected/97f023e7-bed4-4556-8279-20e645880b88-kube-api-access-gt5qn\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:32.090542 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090549 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.090778 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090579 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-bound-sa-token\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.090778 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090608 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ed0c96e-8722-4474-9499-d56cb6a3df9c-trusted-ca\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.090778 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090682 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fz59\" (UniqueName: \"kubernetes.io/projected/a6dc3cee-9579-4833-9632-1c03581ee3b7-kube-api-access-9fz59\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:32.090778 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090767 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.090950 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090826 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-installation-pull-secrets\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.090950 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090856 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/97f023e7-bed4-4556-8279-20e645880b88-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:32.090950 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090884 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-default-certificate\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.090950 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.090860 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:41:32.090950 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.090912 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ed0c96e-8722-4474-9499-d56cb6a3df9c-serving-cert\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.090950 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.090919 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8c4ff68f-n4x6c: secret "image-registry-tls" not found Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.091002 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls podName:9d22ec60-8779-4a14-90f2-e8abff8d6b61 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:32.590978291 +0000 UTC m=+36.740125692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls") pod "image-registry-b8c4ff68f-n4x6c" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61") : secret "image-registry-tls" not found Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091002 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtc5w\" (UniqueName: \"kubernetes.io/projected/9298287b-3797-48a7-995b-f613902a2a98-kube-api-access-wtc5w\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091041 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-trusted-ca\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091074 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/19467e99-0315-4947-9934-80d423a3e33d-snapshots\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091106 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmlnj\" (UniqueName: \"kubernetes.io/projected/19467e99-0315-4947-9934-80d423a3e33d-kube-api-access-pmlnj\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091135 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-758rz\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-kube-api-access-758rz\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091161 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19467e99-0315-4947-9934-80d423a3e33d-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091194 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:32.091228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091225 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7v4g\" (UniqueName: \"kubernetes.io/projected/9ed0c96e-8722-4474-9499-d56cb6a3df9c-kube-api-access-z7v4g\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091249 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-stats-auth\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091318 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ed0c96e-8722-4474-9499-d56cb6a3df9c-config\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091364 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19467e99-0315-4947-9934-80d423a3e33d-serving-cert\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091406 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-certificates\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091441 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d22ec60-8779-4a14-90f2-e8abff8d6b61-ca-trust-extracted\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091482 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-image-registry-private-configuration\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091511 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19467e99-0315-4947-9934-80d423a3e33d-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091588 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19467e99-0315-4947-9934-80d423a3e33d-tmp\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091632 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091682 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6wlr\" (UniqueName: \"kubernetes.io/projected/9d1b0622-6a48-4266-8066-8db5560a77b2-kube-api-access-v6wlr\") pod \"volume-data-source-validator-5c4db8bdf9-zkz22\" (UID: \"9d1b0622-6a48-4266-8066-8db5560a77b2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22" Sep 17 17:41:32.091731 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091717 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:32.092245 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091793 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/97f023e7-bed4-4556-8279-20e645880b88-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:32.092245 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.091838 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:32.092245 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.091893 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19467e99-0315-4947-9934-80d423a3e33d-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.092245 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.091904 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls podName:97f023e7-bed4-4556-8279-20e645880b88 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:32.591886425 +0000 UTC m=+36.741033838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-f4hg9" (UID: "97f023e7-bed4-4556-8279-20e645880b88") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:32.092450 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.092102 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-trusted-ca\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.092450 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.092125 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-certificates\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.092450 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.092221 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19467e99-0315-4947-9934-80d423a3e33d-tmp\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.092450 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.092254 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d22ec60-8779-4a14-90f2-e8abff8d6b61-ca-trust-extracted\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.092725 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.092695 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/19467e99-0315-4947-9934-80d423a3e33d-snapshots\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.092855 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.092777 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19467e99-0315-4947-9934-80d423a3e33d-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.095907 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.095872 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19467e99-0315-4947-9934-80d423a3e33d-serving-cert\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.096023 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.095977 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-installation-pull-secrets\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.096023 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.095990 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-image-registry-private-configuration\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.096493 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.096469 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22"] Sep 17 17:41:32.096639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.096603 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.100910 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.100886 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt"] Sep 17 17:41:32.101057 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.101019 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:41:32.101117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.101079 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 17 17:41:32.101295 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.101282 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:41:32.102216 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.102192 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 17 17:41:32.102387 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.102223 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 17 17:41:32.102387 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.102238 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 17 17:41:32.102520 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.102506 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-b5crx\"" Sep 17 17:41:32.104010 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.103785 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 17:41:32.104010 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.103798 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hq5mj\"" Sep 17 17:41:32.104010 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.103607 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gwk7t\"" Sep 17 17:41:32.104211 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.104056 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 17:41:32.105094 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.104560 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 17:41:32.105980 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.105955 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-bound-sa-token\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.106063 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.106029 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-758rz\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-kube-api-access-758rz\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.106114 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.106073 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt5qn\" (UniqueName: \"kubernetes.io/projected/97f023e7-bed4-4556-8279-20e645880b88-kube-api-access-gt5qn\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:32.107788 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.107771 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmlnj\" (UniqueName: \"kubernetes.io/projected/19467e99-0315-4947-9934-80d423a3e33d-kube-api-access-pmlnj\") pod \"insights-operator-6f9c644dd9-lzhcp\" (UID: \"19467e99-0315-4947-9934-80d423a3e33d\") " pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.143230 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.143195 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7"] Sep 17 17:41:32.143371 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.143319 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt" Sep 17 17:41:32.145819 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.145786 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-58vgj\"" Sep 17 17:41:32.169019 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.168983 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dpn22"] Sep 17 17:41:32.169192 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.169128 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:32.171841 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.171814 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 17 17:41:32.172001 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.171924 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-n48lm\"" Sep 17 17:41:32.172094 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.172079 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 17 17:41:32.192493 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192448 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvntz\" (UniqueName: \"kubernetes.io/projected/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-kube-api-access-jvntz\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.192685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192506 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.192685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192536 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed64f380-f001-47b9-917e-990dd6ea473f-tmp-dir\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.192685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192579 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.192685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192630 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wlr\" (UniqueName: \"kubernetes.io/projected/9d1b0622-6a48-4266-8066-8db5560a77b2-kube-api-access-v6wlr\") pod \"volume-data-source-validator-5c4db8bdf9-zkz22\" (UID: \"9d1b0622-6a48-4266-8066-8db5560a77b2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22" Sep 17 17:41:32.192685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192675 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.192913 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192703 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.192913 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192731 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ed0c96e-8722-4474-9499-d56cb6a3df9c-trusted-ca\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.192913 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192756 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fz59\" (UniqueName: \"kubernetes.io/projected/a6dc3cee-9579-4833-9632-1c03581ee3b7-kube-api-access-9fz59\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:32.192913 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192795 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-default-certificate\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.192913 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192821 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fjch\" (UniqueName: \"kubernetes.io/projected/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-kube-api-access-9fjch\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.192913 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192851 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-serving-cert\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.192913 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192889 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ed0c96e-8722-4474-9499-d56cb6a3df9c-serving-cert\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.193128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192916 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc5w\" (UniqueName: \"kubernetes.io/projected/9298287b-3797-48a7-995b-f613902a2a98-kube-api-access-wtc5w\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.193128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192954 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcn5q\" (UniqueName: \"kubernetes.io/projected/ed64f380-f001-47b9-917e-990dd6ea473f-kube-api-access-xcn5q\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.193128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.192988 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:32.193128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.193020 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7v4g\" (UniqueName: \"kubernetes.io/projected/9ed0c96e-8722-4474-9499-d56cb6a3df9c-kube-api-access-z7v4g\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.193128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.193046 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-stats-auth\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.193128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.193090 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ed0c96e-8722-4474-9499-d56cb6a3df9c-config\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.193128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.193124 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ed64f380-f001-47b9-917e-990dd6ea473f-config-volume\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.193329 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.193158 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-config\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.193329 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.193205 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.193411 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.193374 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:32.693354764 +0000 UTC m=+36.842502177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : configmap references non-existent config key: service-ca.crt Sep 17 17:41:32.193981 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.193964 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:41:32.194026 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.194017 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:32.694001168 +0000 UTC m=+36.843148580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : secret "router-metrics-certs-default" not found Sep 17 17:41:32.195331 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.195298 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ed0c96e-8722-4474-9499-d56cb6a3df9c-trusted-ca\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.195453 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.195398 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9ed0c96e-8722-4474-9499-d56cb6a3df9c-config\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.195538 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.195517 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:41:32.195694 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.195591 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls podName:a6dc3cee-9579-4833-9632-1c03581ee3b7 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:32.695567773 +0000 UTC m=+36.844715188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-ghqgb" (UID: "a6dc3cee-9579-4833-9632-1c03581ee3b7") : secret "samples-operator-tls" not found Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197387 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-lzhcp"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197431 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-wb4t7"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197455 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197481 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b6cb8c868-qgggf"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197496 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197515 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197531 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197558 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197580 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sshtr"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197641 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dpn22"] Sep 17 17:41:32.197769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.197679 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dpn22" Sep 17 17:41:32.200815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.198062 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-stats-auth\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.200815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.198099 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9ed0c96e-8722-4474-9499-d56cb6a3df9c-serving-cert\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.200815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.200118 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 17:41:32.200815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.200470 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-default-certificate\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.200815 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.200662 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcr8x\"" Sep 17 17:41:32.201123 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.200938 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 17:41:32.203284 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.203109 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 17:41:32.204095 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.203888 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6wlr\" (UniqueName: \"kubernetes.io/projected/9d1b0622-6a48-4266-8066-8db5560a77b2-kube-api-access-v6wlr\") pod \"volume-data-source-validator-5c4db8bdf9-zkz22\" (UID: \"9d1b0622-6a48-4266-8066-8db5560a77b2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22" Sep 17 17:41:32.204349 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.204326 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7v4g\" (UniqueName: \"kubernetes.io/projected/9ed0c96e-8722-4474-9499-d56cb6a3df9c-kube-api-access-z7v4g\") pod \"console-operator-fd4bb46c-wb4t7\" (UID: \"9ed0c96e-8722-4474-9499-d56cb6a3df9c\") " pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.206891 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.206864 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fz59\" (UniqueName: \"kubernetes.io/projected/a6dc3cee-9579-4833-9632-1c03581ee3b7-kube-api-access-9fz59\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:32.208489 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.208466 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtc5w\" (UniqueName: \"kubernetes.io/projected/9298287b-3797-48a7-995b-f613902a2a98-kube-api-access-wtc5w\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.237278 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.237245 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" Sep 17 17:41:32.259034 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.259000 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:32.294027 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.293989 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ed64f380-f001-47b9-917e-990dd6ea473f-config-volume\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.294222 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294047 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-config\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.294222 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294104 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.294222 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294134 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d09443ff-a2b4-4a53-861f-810d4d2cd4f3-cert\") pod \"ingress-canary-dpn22\" (UID: \"d09443ff-a2b4-4a53-861f-810d4d2cd4f3\") " pod="openshift-ingress-canary/ingress-canary-dpn22" Sep 17 17:41:32.294222 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294168 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp7x9\" (UniqueName: \"kubernetes.io/projected/d09443ff-a2b4-4a53-861f-810d4d2cd4f3-kube-api-access-gp7x9\") pod \"ingress-canary-dpn22\" (UID: \"d09443ff-a2b4-4a53-861f-810d4d2cd4f3\") " pod="openshift-ingress-canary/ingress-canary-dpn22" Sep 17 17:41:32.294222 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294202 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvntz\" (UniqueName: \"kubernetes.io/projected/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-kube-api-access-jvntz\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.294483 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294229 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e37515af-75c0-4ee7-8552-5212fc707e1f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:32.294483 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294272 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed64f380-f001-47b9-917e-990dd6ea473f-tmp-dir\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.294483 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294317 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.294483 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294367 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.294483 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294413 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:32.294483 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294454 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fjch\" (UniqueName: \"kubernetes.io/projected/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-kube-api-access-9fjch\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.294483 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294472 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-serving-cert\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.294847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294495 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk85f\" (UniqueName: \"kubernetes.io/projected/c85a02d3-2fba-4bde-b739-2486d2a637a4-kube-api-access-vk85f\") pod \"network-check-source-7cbc45fddc-wt2kt\" (UID: \"c85a02d3-2fba-4bde-b739-2486d2a637a4\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt" Sep 17 17:41:32.294847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294519 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcn5q\" (UniqueName: \"kubernetes.io/projected/ed64f380-f001-47b9-917e-990dd6ea473f-kube-api-access-xcn5q\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.294847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294703 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ed64f380-f001-47b9-917e-990dd6ea473f-config-volume\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.294847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294731 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-config\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.294847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.294768 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.295063 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.295011 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:41:32.295109 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.295081 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls podName:ed64f380-f001-47b9-917e-990dd6ea473f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:32.795062089 +0000 UTC m=+36.944209502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls") pod "dns-default-sshtr" (UID: "ed64f380-f001-47b9-917e-990dd6ea473f") : secret "dns-default-metrics-tls" not found Sep 17 17:41:32.295339 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.295319 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed64f380-f001-47b9-917e-990dd6ea473f-tmp-dir\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.297374 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.297346 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.297535 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.297426 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-serving-cert\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.304401 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.304375 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvntz\" (UniqueName: \"kubernetes.io/projected/ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8-kube-api-access-jvntz\") pod \"service-ca-operator-57cdc4b768-fslx6\" (UID: \"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.304547 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.304468 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcn5q\" (UniqueName: \"kubernetes.io/projected/ed64f380-f001-47b9-917e-990dd6ea473f-kube-api-access-xcn5q\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.304702 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.304686 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fjch\" (UniqueName: \"kubernetes.io/projected/778a9f61-f31c-4157-9e8c-69c40f0dd7a3-kube-api-access-9fjch\") pod \"kube-storage-version-migrator-operator-55889f7b47-nhxzc\" (UID: \"778a9f61-f31c-4157-9e8c-69c40f0dd7a3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.321679 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.321643 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22" Sep 17 17:41:32.377191 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.377151 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" Sep 17 17:41:32.395534 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.395503 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d09443ff-a2b4-4a53-861f-810d4d2cd4f3-cert\") pod \"ingress-canary-dpn22\" (UID: \"d09443ff-a2b4-4a53-861f-810d4d2cd4f3\") " pod="openshift-ingress-canary/ingress-canary-dpn22" Sep 17 17:41:32.395646 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.395535 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp7x9\" (UniqueName: \"kubernetes.io/projected/d09443ff-a2b4-4a53-861f-810d4d2cd4f3-kube-api-access-gp7x9\") pod \"ingress-canary-dpn22\" (UID: \"d09443ff-a2b4-4a53-861f-810d4d2cd4f3\") " pod="openshift-ingress-canary/ingress-canary-dpn22" Sep 17 17:41:32.395717 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.395663 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e37515af-75c0-4ee7-8552-5212fc707e1f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:32.395788 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.395765 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:32.395837 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.395819 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk85f\" (UniqueName: \"kubernetes.io/projected/c85a02d3-2fba-4bde-b739-2486d2a637a4-kube-api-access-vk85f\") pod \"network-check-source-7cbc45fddc-wt2kt\" (UID: \"c85a02d3-2fba-4bde-b739-2486d2a637a4\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt" Sep 17 17:41:32.395935 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.395912 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:41:32.396023 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.396002 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert podName:e37515af-75c0-4ee7-8552-5212fc707e1f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:32.895982264 +0000 UTC m=+37.045129662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-sp2x7" (UID: "e37515af-75c0-4ee7-8552-5212fc707e1f") : secret "networking-console-plugin-cert" not found Sep 17 17:41:32.396284 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.396268 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e37515af-75c0-4ee7-8552-5212fc707e1f-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:32.398141 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.398119 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d09443ff-a2b4-4a53-861f-810d4d2cd4f3-cert\") pod \"ingress-canary-dpn22\" (UID: \"d09443ff-a2b4-4a53-861f-810d4d2cd4f3\") " pod="openshift-ingress-canary/ingress-canary-dpn22" Sep 17 17:41:32.405273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.405254 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk85f\" (UniqueName: \"kubernetes.io/projected/c85a02d3-2fba-4bde-b739-2486d2a637a4-kube-api-access-vk85f\") pod \"network-check-source-7cbc45fddc-wt2kt\" (UID: \"c85a02d3-2fba-4bde-b739-2486d2a637a4\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt" Sep 17 17:41:32.405326 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.405308 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp7x9\" (UniqueName: \"kubernetes.io/projected/d09443ff-a2b4-4a53-861f-810d4d2cd4f3-kube-api-access-gp7x9\") pod \"ingress-canary-dpn22\" (UID: \"d09443ff-a2b4-4a53-861f-810d4d2cd4f3\") " pod="openshift-ingress-canary/ingress-canary-dpn22" Sep 17 17:41:32.428187 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.428155 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" Sep 17 17:41:32.452528 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.452460 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt" Sep 17 17:41:32.519772 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.515758 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dpn22" Sep 17 17:41:32.599320 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.598250 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:32.599320 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.598332 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:32.599320 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.598524 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:41:32.599320 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.598543 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8c4ff68f-n4x6c: secret "image-registry-tls" not found Sep 17 17:41:32.599320 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.598609 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls podName:9d22ec60-8779-4a14-90f2-e8abff8d6b61 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:33.598591289 +0000 UTC m=+37.747738707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls") pod "image-registry-b8c4ff68f-n4x6c" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61") : secret "image-registry-tls" not found Sep 17 17:41:32.599320 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.599252 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:32.599855 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.599329 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls podName:97f023e7-bed4-4556-8279-20e645880b88 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:33.599309809 +0000 UTC m=+37.748457227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-f4hg9" (UID: "97f023e7-bed4-4556-8279-20e645880b88") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:32.699631 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.699587 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.699798 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.699704 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:32.699798 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.699750 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:41:32.699798 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.699780 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:32.699967 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.699821 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:33.699802091 +0000 UTC m=+37.848949488 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : secret "router-metrics-certs-default" not found Sep 17 17:41:32.699967 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.699839 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:41:32.699967 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.699885 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls podName:a6dc3cee-9579-4833-9632-1c03581ee3b7 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:33.699873201 +0000 UTC m=+37.849020636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-ghqgb" (UID: "a6dc3cee-9579-4833-9632-1c03581ee3b7") : secret "samples-operator-tls" not found Sep 17 17:41:32.699967 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.699905 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:33.699891089 +0000 UTC m=+37.849038516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : configmap references non-existent config key: service-ca.crt Sep 17 17:41:32.798363 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.798279 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6"] Sep 17 17:41:32.800262 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.800237 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22"] Sep 17 17:41:32.800492 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.800466 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:32.800634 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.800604 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:41:32.800691 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.800680 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls podName:ed64f380-f001-47b9-917e-990dd6ea473f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:33.800665594 +0000 UTC m=+37.949812992 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls") pod "dns-default-sshtr" (UID: "ed64f380-f001-47b9-917e-990dd6ea473f") : secret "dns-default-metrics-tls" not found Sep 17 17:41:32.805607 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.805583 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc"] Sep 17 17:41:32.807078 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.806900 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-lzhcp"] Sep 17 17:41:32.807910 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.807891 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-wb4t7"] Sep 17 17:41:32.808703 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.808681 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt"] Sep 17 17:41:32.839018 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.838979 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dpn22"] Sep 17 17:41:32.867022 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:32.866982 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea9c4c8f_28b5_4f82_a0f0_cf1c7dfe1ff8.slice/crio-dd1e905cbfbec08ad26516d0333e23186dc30e8737739ba3b9c9049a4d3adfeb WatchSource:0}: Error finding container dd1e905cbfbec08ad26516d0333e23186dc30e8737739ba3b9c9049a4d3adfeb: Status 404 returned error can't find the container with id dd1e905cbfbec08ad26516d0333e23186dc30e8737739ba3b9c9049a4d3adfeb Sep 17 17:41:32.867658 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:32.867611 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1b0622_6a48_4266_8066_8db5560a77b2.slice/crio-367986da16c4214d613945dcf6283e5a2002cdb0dbb10dcde06a5e61697c1bda WatchSource:0}: Error finding container 367986da16c4214d613945dcf6283e5a2002cdb0dbb10dcde06a5e61697c1bda: Status 404 returned error can't find the container with id 367986da16c4214d613945dcf6283e5a2002cdb0dbb10dcde06a5e61697c1bda Sep 17 17:41:32.868603 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:32.868581 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19467e99_0315_4947_9934_80d423a3e33d.slice/crio-adbd452be3208ebae63712c9b94fbdcf02650ad052963f680272e6ca6d102f89 WatchSource:0}: Error finding container adbd452be3208ebae63712c9b94fbdcf02650ad052963f680272e6ca6d102f89: Status 404 returned error can't find the container with id adbd452be3208ebae63712c9b94fbdcf02650ad052963f680272e6ca6d102f89 Sep 17 17:41:32.901232 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:32.901197 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:32.901410 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.901390 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:41:32.901465 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:32.901456 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert podName:e37515af-75c0-4ee7-8552-5212fc707e1f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:33.901442169 +0000 UTC m=+38.050589567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-sp2x7" (UID: "e37515af-75c0-4ee7-8552-5212fc707e1f") : secret "networking-console-plugin-cert" not found Sep 17 17:41:33.032695 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.032668 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:41:33.038045 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.038023 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 17:41:33.241973 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.241928 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22" event={"ID":"9d1b0622-6a48-4266-8066-8db5560a77b2","Type":"ContainerStarted","Data":"367986da16c4214d613945dcf6283e5a2002cdb0dbb10dcde06a5e61697c1bda"} Sep 17 17:41:33.243116 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.243086 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" event={"ID":"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8","Type":"ContainerStarted","Data":"dd1e905cbfbec08ad26516d0333e23186dc30e8737739ba3b9c9049a4d3adfeb"} Sep 17 17:41:33.244196 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.244172 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" event={"ID":"19467e99-0315-4947-9934-80d423a3e33d","Type":"ContainerStarted","Data":"adbd452be3208ebae63712c9b94fbdcf02650ad052963f680272e6ca6d102f89"} Sep 17 17:41:33.245186 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.245164 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" event={"ID":"778a9f61-f31c-4157-9e8c-69c40f0dd7a3","Type":"ContainerStarted","Data":"7ec2335a4a111b749d635cae9e7912e4c9d428f2895e0122a7a11fafc19e1ae1"} Sep 17 17:41:33.248198 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.248172 2586 generic.go:356] "Generic (PLEG): container finished" podID="9eedb140-05c7-48ce-8171-13998ef8be9e" containerID="3c49cf2a0b3e3ab66792051cdb869025beed6b1875b0d3ead1471ef572d63b29" exitCode=0 Sep 17 17:41:33.248268 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.248249 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" event={"ID":"9eedb140-05c7-48ce-8171-13998ef8be9e","Type":"ContainerDied","Data":"3c49cf2a0b3e3ab66792051cdb869025beed6b1875b0d3ead1471ef572d63b29"} Sep 17 17:41:33.249514 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.249491 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt" event={"ID":"c85a02d3-2fba-4bde-b739-2486d2a637a4","Type":"ContainerStarted","Data":"8cdc9a7ee2204ae656f556c23b527a8f7c2e01807adbe7a0c3717ae3ef5e6e6f"} Sep 17 17:41:33.250659 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.250639 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" event={"ID":"9ed0c96e-8722-4474-9499-d56cb6a3df9c","Type":"ContainerStarted","Data":"23fb09cda13bdde61b56e06ea583975bcb582b7296a98951bd634b444645f03e"} Sep 17 17:41:33.254979 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.254954 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dpn22" event={"ID":"d09443ff-a2b4-4a53-861f-810d4d2cd4f3","Type":"ContainerStarted","Data":"99cb3e399f3c73d70dd9f025805d174b0f84d8a8d76f8dec23c21fd845db3e61"} Sep 17 17:41:33.608500 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.608461 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:33.608685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.608563 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:33.609203 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.608773 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:41:33.609203 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.608794 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8c4ff68f-n4x6c: secret "image-registry-tls" not found Sep 17 17:41:33.609203 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.608849 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls podName:9d22ec60-8779-4a14-90f2-e8abff8d6b61 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:35.608831923 +0000 UTC m=+39.757979341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls") pod "image-registry-b8c4ff68f-n4x6c" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61") : secret "image-registry-tls" not found Sep 17 17:41:33.609203 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.609108 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:33.609203 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.609189 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls podName:97f023e7-bed4-4556-8279-20e645880b88 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:35.609172366 +0000 UTC m=+39.758319778 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-f4hg9" (UID: "97f023e7-bed4-4556-8279-20e645880b88") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:33.709332 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.709279 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:33.709582 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.709357 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:33.709582 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.709407 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:33.709582 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.709519 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:41:33.709582 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.709570 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls podName:a6dc3cee-9579-4833-9632-1c03581ee3b7 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:35.709556149 +0000 UTC m=+39.858703546 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-ghqgb" (UID: "a6dc3cee-9579-4833-9632-1c03581ee3b7") : secret "samples-operator-tls" not found Sep 17 17:41:33.710021 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.709779 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:41:33.710021 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.709840 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:35.709824877 +0000 UTC m=+39.858972275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : secret "router-metrics-certs-default" not found Sep 17 17:41:33.710021 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.709860 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:35.70984968 +0000 UTC m=+39.858997084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : configmap references non-existent config key: service-ca.crt Sep 17 17:41:33.810706 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.810665 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:33.810905 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.810874 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:41:33.810967 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.810940 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls podName:ed64f380-f001-47b9-917e-990dd6ea473f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:35.810919459 +0000 UTC m=+39.960066871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls") pod "dns-default-sshtr" (UID: "ed64f380-f001-47b9-917e-990dd6ea473f") : secret "dns-default-metrics-tls" not found Sep 17 17:41:33.911482 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:33.911393 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:33.912182 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.911746 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:41:33.912182 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:33.911831 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert podName:e37515af-75c0-4ee7-8552-5212fc707e1f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:35.911811167 +0000 UTC m=+40.060958568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-sp2x7" (UID: "e37515af-75c0-4ee7-8552-5212fc707e1f") : secret "networking-console-plugin-cert" not found Sep 17 17:41:34.268120 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:34.267042 2586 generic.go:356] "Generic (PLEG): container finished" podID="9eedb140-05c7-48ce-8171-13998ef8be9e" containerID="37de0eb6f74d0f9733a7a84aeabf0a159c6c34d95f71b8c24435826aa498e014" exitCode=0 Sep 17 17:41:34.268120 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:34.267111 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" event={"ID":"9eedb140-05c7-48ce-8171-13998ef8be9e","Type":"ContainerDied","Data":"37de0eb6f74d0f9733a7a84aeabf0a159c6c34d95f71b8c24435826aa498e014"} Sep 17 17:41:35.280187 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:35.280150 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" event={"ID":"9eedb140-05c7-48ce-8171-13998ef8be9e","Type":"ContainerStarted","Data":"02c6826a6383957b1ca1024b55cebd070ba0a2b7c55644d1bd1e99fd131a3811"} Sep 17 17:41:35.634848 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:35.634062 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:35.634848 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:35.634134 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:35.634848 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.634277 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:41:35.634848 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.634294 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8c4ff68f-n4x6c: secret "image-registry-tls" not found Sep 17 17:41:35.634848 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.634350 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls podName:9d22ec60-8779-4a14-90f2-e8abff8d6b61 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:39.63433662 +0000 UTC m=+43.783484018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls") pod "image-registry-b8c4ff68f-n4x6c" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61") : secret "image-registry-tls" not found Sep 17 17:41:35.634848 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.634772 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:35.634848 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.634820 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls podName:97f023e7-bed4-4556-8279-20e645880b88 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:39.634805616 +0000 UTC m=+43.783953018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-f4hg9" (UID: "97f023e7-bed4-4556-8279-20e645880b88") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:35.734950 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:35.734906 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:35.735096 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.735053 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:41:35.735146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:35.735113 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:35.735146 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.735133 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls podName:a6dc3cee-9579-4833-9632-1c03581ee3b7 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:39.735112391 +0000 UTC m=+43.884259794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-ghqgb" (UID: "a6dc3cee-9579-4833-9632-1c03581ee3b7") : secret "samples-operator-tls" not found Sep 17 17:41:35.735239 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.735224 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:39.735205244 +0000 UTC m=+43.884352666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : configmap references non-existent config key: service-ca.crt Sep 17 17:41:35.735285 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:35.735254 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:35.735411 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.735395 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:41:35.735554 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.735433 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:39.735421445 +0000 UTC m=+43.884568858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : secret "router-metrics-certs-default" not found Sep 17 17:41:35.836674 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:35.836630 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:35.836901 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.836881 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:41:35.836976 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.836958 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls podName:ed64f380-f001-47b9-917e-990dd6ea473f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:39.83693904 +0000 UTC m=+43.986086445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls") pod "dns-default-sshtr" (UID: "ed64f380-f001-47b9-917e-990dd6ea473f") : secret "dns-default-metrics-tls" not found Sep 17 17:41:35.937721 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:35.937635 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:35.937902 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.937827 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:41:35.937974 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:35.937920 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert podName:e37515af-75c0-4ee7-8552-5212fc707e1f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:39.937898394 +0000 UTC m=+44.087045816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-sp2x7" (UID: "e37515af-75c0-4ee7-8552-5212fc707e1f") : secret "networking-console-plugin-cert" not found Sep 17 17:41:39.670782 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:39.670732 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:39.671272 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:39.670818 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:39.671272 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.671041 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:41:39.671272 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.671056 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8c4ff68f-n4x6c: secret "image-registry-tls" not found Sep 17 17:41:39.671272 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.671109 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls podName:9d22ec60-8779-4a14-90f2-e8abff8d6b61 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:47.67109463 +0000 UTC m=+51.820242028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls") pod "image-registry-b8c4ff68f-n4x6c" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61") : secret "image-registry-tls" not found Sep 17 17:41:39.671549 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.671524 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:39.671602 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.671586 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls podName:97f023e7-bed4-4556-8279-20e645880b88 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:47.671572109 +0000 UTC m=+51.820719521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-f4hg9" (UID: "97f023e7-bed4-4556-8279-20e645880b88") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:39.772037 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:39.771991 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:39.772218 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:39.772090 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:39.772218 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:39.772151 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:39.772218 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.772175 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:41:39.772323 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.772247 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:41:39.772323 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.772254 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls podName:a6dc3cee-9579-4833-9632-1c03581ee3b7 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:47.772232069 +0000 UTC m=+51.921379471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-ghqgb" (UID: "a6dc3cee-9579-4833-9632-1c03581ee3b7") : secret "samples-operator-tls" not found Sep 17 17:41:39.772323 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.772288 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:47.772271472 +0000 UTC m=+51.921418890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : configmap references non-existent config key: service-ca.crt Sep 17 17:41:39.772323 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.772317 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:41:47.772305734 +0000 UTC m=+51.921453133 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : secret "router-metrics-certs-default" not found Sep 17 17:41:39.873453 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:39.873419 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:39.873671 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.873590 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:41:39.873752 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.873679 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls podName:ed64f380-f001-47b9-917e-990dd6ea473f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:47.873663449 +0000 UTC m=+52.022810847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls") pod "dns-default-sshtr" (UID: "ed64f380-f001-47b9-917e-990dd6ea473f") : secret "dns-default-metrics-tls" not found Sep 17 17:41:39.974446 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:39.974362 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:39.974604 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.974521 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:41:39.974604 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:39.974595 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert podName:e37515af-75c0-4ee7-8552-5212fc707e1f nodeName:}" failed. No retries permitted until 2026-09-17 17:41:47.974579409 +0000 UTC m=+52.123726807 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-sp2x7" (UID: "e37515af-75c0-4ee7-8552-5212fc707e1f") : secret "networking-console-plugin-cert" not found Sep 17 17:41:41.305256 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.305146 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt" event={"ID":"c85a02d3-2fba-4bde-b739-2486d2a637a4","Type":"ContainerStarted","Data":"5b9d7f6c199c03690916d78dbea18a418b0464bc76ca7470ff7608bbd4c7888c"} Sep 17 17:41:41.307875 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.307848 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/0.log" Sep 17 17:41:41.308117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.307902 2586 generic.go:356] "Generic (PLEG): container finished" podID="9ed0c96e-8722-4474-9499-d56cb6a3df9c" containerID="c2110fb637747aa17e8cce2815e285110eadd06c2d06bd6407262e5aa087ff21" exitCode=255 Sep 17 17:41:41.308117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.307984 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" event={"ID":"9ed0c96e-8722-4474-9499-d56cb6a3df9c","Type":"ContainerDied","Data":"c2110fb637747aa17e8cce2815e285110eadd06c2d06bd6407262e5aa087ff21"} Sep 17 17:41:41.308351 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.308261 2586 scope.go:117] "RemoveContainer" containerID="c2110fb637747aa17e8cce2815e285110eadd06c2d06bd6407262e5aa087ff21" Sep 17 17:41:41.310813 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.310566 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dpn22" event={"ID":"d09443ff-a2b4-4a53-861f-810d4d2cd4f3","Type":"ContainerStarted","Data":"159970ca58b30361015b78032da5376e2cd21a7658fa79cb5ab3d06ce7c7bae5"} Sep 17 17:41:41.312405 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.312223 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22" event={"ID":"9d1b0622-6a48-4266-8066-8db5560a77b2","Type":"ContainerStarted","Data":"6c6e26c25f54a498ea01e774f7906a87c2b7da62d1ac324bc08e87a82f3b158e"} Sep 17 17:41:41.314965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.314938 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" event={"ID":"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8","Type":"ContainerStarted","Data":"b2087acd8f48aa23621f45e58f422a581da5ff350f5bcc97b7f09b0c83ce1bca"} Sep 17 17:41:41.316949 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.316929 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" event={"ID":"19467e99-0315-4947-9934-80d423a3e33d","Type":"ContainerStarted","Data":"20df0b9f873df472ce701d15ee418feda3e5bb85e2bbbd50b96d6304a05df483"} Sep 17 17:41:41.318884 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.318861 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" event={"ID":"778a9f61-f31c-4157-9e8c-69c40f0dd7a3","Type":"ContainerStarted","Data":"e1de97cbe28a4648a8367e813220624edf1372efb87362250de6dcbf17ce22d9"} Sep 17 17:41:41.322372 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.322321 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pcvrb" podStartSLOduration=14.236700242 podStartE2EDuration="43.322304551s" totalImagesPullingTime="29.085604309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:35.305572993 +0000 UTC m=+39.454720413" watchObservedRunningTime="2026-09-17 17:41:41.322304551 +0000 UTC m=+45.471451969" Sep 17 17:41:41.322856 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.322815 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-wt2kt" podStartSLOduration=7.577610271 podStartE2EDuration="15.322804963s" totalImagesPullingTime="7.745194692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:41.322020518 +0000 UTC m=+45.471167944" watchObservedRunningTime="2026-09-17 17:41:41.322804963 +0000 UTC m=+45.471952385" Sep 17 17:41:41.341586 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.341534 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" podStartSLOduration=14.59620073 podStartE2EDuration="22.34151678s" totalImagesPullingTime="7.74531605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:41.340706878 +0000 UTC m=+45.489854298" watchObservedRunningTime="2026-09-17 17:41:41.34151678 +0000 UTC m=+45.490664211" Sep 17 17:41:41.362169 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.362011 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" podStartSLOduration=10.616158843000001 podStartE2EDuration="18.36198872s" totalImagesPullingTime="7.745829877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:41.361904739 +0000 UTC m=+45.511052161" watchObservedRunningTime="2026-09-17 17:41:41.36198872 +0000 UTC m=+45.511136135" Sep 17 17:41:41.400485 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.400416 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-zkz22" podStartSLOduration=15.117122784 podStartE2EDuration="22.400380793s" totalImagesPullingTime="7.283258009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:41.398754584 +0000 UTC m=+45.547902006" watchObservedRunningTime="2026-09-17 17:41:41.400380793 +0000 UTC m=+45.549528217" Sep 17 17:41:41.400678 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.400598 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" podStartSLOduration=8.612727718 podStartE2EDuration="16.400593187s" totalImagesPullingTime="7.787865469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:41.380083607 +0000 UTC m=+45.529231032" watchObservedRunningTime="2026-09-17 17:41:41.400593187 +0000 UTC m=+45.549740611" Sep 17 17:41:41.437748 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:41.436760 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dpn22" podStartSLOduration=2.6825385710000003 podStartE2EDuration="10.436740871s" totalImagesPullingTime="7.7542023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:41.418118643 +0000 UTC m=+45.567266064" watchObservedRunningTime="2026-09-17 17:41:41.436740871 +0000 UTC m=+45.585888292" Sep 17 17:41:42.149232 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.149197 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g8l44"] Sep 17 17:41:42.152014 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.151988 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.155315 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.155287 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 17:41:42.155315 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.155301 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cmvbs\"" Sep 17 17:41:42.155802 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.155782 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 17:41:42.171085 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.171056 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g8l44"] Sep 17 17:41:42.259227 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.259183 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:42.259391 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.259247 2586 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:42.296938 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.296906 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-crio-socket\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.297114 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.296960 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.297114 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.297068 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plv5z\" (UniqueName: \"kubernetes.io/projected/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-api-access-plv5z\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.297114 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.297112 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.297238 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.297139 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-data-volume\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.325113 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.325089 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:41:42.325665 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.325648 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/0.log" Sep 17 17:41:42.325722 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.325685 2586 generic.go:356] "Generic (PLEG): container finished" podID="9ed0c96e-8722-4474-9499-d56cb6a3df9c" containerID="f51dda88f615d0b1abffac79c7081acbc8981c0075bfbbb0130cd2dfd4437b5c" exitCode=255 Sep 17 17:41:42.325836 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.325813 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" event={"ID":"9ed0c96e-8722-4474-9499-d56cb6a3df9c","Type":"ContainerDied","Data":"f51dda88f615d0b1abffac79c7081acbc8981c0075bfbbb0130cd2dfd4437b5c"} Sep 17 17:41:42.325892 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.325881 2586 scope.go:117] "RemoveContainer" containerID="c2110fb637747aa17e8cce2815e285110eadd06c2d06bd6407262e5aa087ff21" Sep 17 17:41:42.326157 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.326135 2586 scope.go:117] "RemoveContainer" containerID="f51dda88f615d0b1abffac79c7081acbc8981c0075bfbbb0130cd2dfd4437b5c" Sep 17 17:41:42.326345 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.326325 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-wb4t7_openshift-console-operator(9ed0c96e-8722-4474-9499-d56cb6a3df9c)\"" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" podUID="9ed0c96e-8722-4474-9499-d56cb6a3df9c" Sep 17 17:41:42.398265 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.398224 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-crio-socket\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.398452 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.398285 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.398452 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.398370 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plv5z\" (UniqueName: \"kubernetes.io/projected/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-api-access-plv5z\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.398452 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.398406 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.398452 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.398406 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:41:42.398452 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.398439 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-data-volume\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.398742 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.398472 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-crio-socket\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.398742 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.398493 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:42.898468439 +0000 UTC m=+47.047615851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : secret "insights-runtime-extractor-tls" not found Sep 17 17:41:42.398742 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.398549 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:41:42.398742 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.398588 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:42.89857626 +0000 UTC m=+47.047723659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : configmap "kube-rbac-proxy" not found Sep 17 17:41:42.398946 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.398807 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-data-volume\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.417891 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.417826 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plv5z\" (UniqueName: \"kubernetes.io/projected/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-api-access-plv5z\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.903469 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.903433 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.903648 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:42.903512 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:42.903648 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.903581 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:41:42.903648 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.903587 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:41:42.903765 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.903664 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:43.903647718 +0000 UTC m=+48.052795115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : secret "insights-runtime-extractor-tls" not found Sep 17 17:41:42.903765 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:42.903679 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:43.903673095 +0000 UTC m=+48.052820493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : configmap "kube-rbac-proxy" not found Sep 17 17:41:43.330736 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:43.330659 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:41:43.331075 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:43.331014 2586 scope.go:117] "RemoveContainer" containerID="f51dda88f615d0b1abffac79c7081acbc8981c0075bfbbb0130cd2dfd4437b5c" Sep 17 17:41:43.331182 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:43.331163 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-wb4t7_openshift-console-operator(9ed0c96e-8722-4474-9499-d56cb6a3df9c)\"" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" podUID="9ed0c96e-8722-4474-9499-d56cb6a3df9c" Sep 17 17:41:43.912794 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:43.912760 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:43.912971 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:43.912873 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:43.912971 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:43.912878 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:41:43.912971 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:43.912947 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:45.912925644 +0000 UTC m=+50.062073042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : configmap "kube-rbac-proxy" not found Sep 17 17:41:43.913077 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:43.912986 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:41:43.913077 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:43.913032 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:45.913020459 +0000 UTC m=+50.062167856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : secret "insights-runtime-extractor-tls" not found Sep 17 17:41:43.932978 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:43.932950 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kz957_4d0ec105-91bf-460a-9fcf-307246b66218/dns-node-resolver/0.log" Sep 17 17:41:45.332232 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:45.332202 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tnn9m_6d3c9ef5-2029-476f-bc07-5cc54bc5ad18/node-ca/0.log" Sep 17 17:41:45.929975 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:45.929941 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:45.930171 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:45.930067 2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 17:41:45.930171 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:45.930091 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:45.930171 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:45.930129 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:49.930113552 +0000 UTC m=+54.079260950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : configmap "kube-rbac-proxy" not found Sep 17 17:41:45.930332 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:45.930195 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:41:45.930332 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:45.930256 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:49.930240696 +0000 UTC m=+54.079388095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : secret "insights-runtime-extractor-tls" not found Sep 17 17:41:46.139129 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:46.139103 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dpn22_d09443ff-a2b4-4a53-861f-810d4d2cd4f3/serve-healthcheck-canary/0.log" Sep 17 17:41:47.744758 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:47.744720 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:41:47.745202 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:47.744784 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:41:47.745202 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.744885 2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:47.745202 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.744893 2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 17:41:47.745202 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.744913 2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b8c4ff68f-n4x6c: secret "image-registry-tls" not found Sep 17 17:41:47.745202 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.744971 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls podName:97f023e7-bed4-4556-8279-20e645880b88 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.744951426 +0000 UTC m=+67.894098826 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-f4hg9" (UID: "97f023e7-bed4-4556-8279-20e645880b88") : secret "cluster-monitoring-operator-tls" not found Sep 17 17:41:47.745202 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.744995 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls podName:9d22ec60-8779-4a14-90f2-e8abff8d6b61 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.744985323 +0000 UTC m=+67.894132724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls") pod "image-registry-b8c4ff68f-n4x6c" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61") : secret "image-registry-tls" not found Sep 17 17:41:47.845923 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:47.845884 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:47.846104 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:47.845967 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:41:47.846104 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.846064 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.846044265 +0000 UTC m=+67.995191683 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : configmap references non-existent config key: service-ca.crt Sep 17 17:41:47.846104 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:47.846089 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:41:47.846267 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.846133 2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 17:41:47.846267 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.846196 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs podName:9298287b-3797-48a7-995b-f613902a2a98 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.846178796 +0000 UTC m=+67.995326214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs") pod "router-default-7b6cb8c868-qgggf" (UID: "9298287b-3797-48a7-995b-f613902a2a98") : secret "router-metrics-certs-default" not found Sep 17 17:41:47.846267 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.846208 2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 17:41:47.846267 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.846237 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls podName:a6dc3cee-9579-4833-9632-1c03581ee3b7 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.846228953 +0000 UTC m=+67.995376350 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-ghqgb" (UID: "a6dc3cee-9579-4833-9632-1c03581ee3b7") : secret "samples-operator-tls" not found Sep 17 17:41:47.946749 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:47.946705 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:41:47.947021 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.946857 2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 17:41:47.947021 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:47.946926 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls podName:ed64f380-f001-47b9-917e-990dd6ea473f nodeName:}" failed. No retries permitted until 2026-09-17 17:42:03.9469093 +0000 UTC m=+68.096056698 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls") pod "dns-default-sshtr" (UID: "ed64f380-f001-47b9-917e-990dd6ea473f") : secret "dns-default-metrics-tls" not found Sep 17 17:41:48.047478 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:48.047394 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:41:48.047645 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:48.047543 2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 17:41:48.047645 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:48.047639 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert podName:e37515af-75c0-4ee7-8552-5212fc707e1f nodeName:}" failed. No retries permitted until 2026-09-17 17:42:04.04760781 +0000 UTC m=+68.196755208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-sp2x7" (UID: "e37515af-75c0-4ee7-8552-5212fc707e1f") : secret "networking-console-plugin-cert" not found Sep 17 17:41:49.965954 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:49.965904 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:49.966394 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:49.966024 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:49.966394 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:49.966076 2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 17:41:49.966394 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:41:49.966168 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls podName:3983db55-61ff-4e21-bfbb-b152cdd0eb9a nodeName:}" failed. No retries permitted until 2026-09-17 17:41:57.966146813 +0000 UTC m=+62.115294211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8l44" (UID: "3983db55-61ff-4e21-bfbb-b152cdd0eb9a") : secret "insights-runtime-extractor-tls" not found Sep 17 17:41:49.966708 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:49.966687 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:52.259610 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:52.259576 2586 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:52.260030 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:52.259649 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:52.260030 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:52.259958 2586 scope.go:117] "RemoveContainer" containerID="f51dda88f615d0b1abffac79c7081acbc8981c0075bfbbb0130cd2dfd4437b5c" Sep 17 17:41:52.361639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:52.361599 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:41:53.367012 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:53.366984 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:41:53.367398 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:53.367058 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" event={"ID":"9ed0c96e-8722-4474-9499-d56cb6a3df9c","Type":"ContainerStarted","Data":"1fe3fb8131864579ed89f34e61591857c436d7911eb4d2c2be164c17d5aa9b12"} Sep 17 17:41:53.367398 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:53.367352 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:53.373544 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:53.373515 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" Sep 17 17:41:53.390001 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:53.389952 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-wb4t7" podStartSLOduration=26.643919719 podStartE2EDuration="34.389936667s" totalImagesPullingTime="7.746016948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:41:53.389480264 +0000 UTC m=+57.538627684" watchObservedRunningTime="2026-09-17 17:41:53.389936667 +0000 UTC m=+57.539084083" Sep 17 17:41:58.041298 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:58.041258 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:58.043658 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:58.043635 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3983db55-61ff-4e21-bfbb-b152cdd0eb9a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8l44\" (UID: \"3983db55-61ff-4e21-bfbb-b152cdd0eb9a\") " pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:58.067874 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:58.067843 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cmvbs\"" Sep 17 17:41:58.073843 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:58.073819 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g8l44" Sep 17 17:41:58.386456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:58.386420 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g8l44"] Sep 17 17:41:58.389830 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:41:58.389797 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3983db55_61ff_4e21_bfbb_b152cdd0eb9a.slice/crio-d2fa65f51c7f51b61889a808ff95fa8f1e8cddd857f373054ab2da98f9bf2291 WatchSource:0}: Error finding container d2fa65f51c7f51b61889a808ff95fa8f1e8cddd857f373054ab2da98f9bf2291: Status 404 returned error can't find the container with id d2fa65f51c7f51b61889a808ff95fa8f1e8cddd857f373054ab2da98f9bf2291 Sep 17 17:41:59.237690 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:59.237657 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6mvdd" Sep 17 17:41:59.387158 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:59.387124 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8l44" event={"ID":"3983db55-61ff-4e21-bfbb-b152cdd0eb9a","Type":"ContainerStarted","Data":"c5b666724f07c7dfbc34ebc2fcb0e704a8fc3f0b49056316af6c2584ea607593"} Sep 17 17:41:59.387328 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:41:59.387180 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8l44" event={"ID":"3983db55-61ff-4e21-bfbb-b152cdd0eb9a","Type":"ContainerStarted","Data":"d2fa65f51c7f51b61889a808ff95fa8f1e8cddd857f373054ab2da98f9bf2291"} Sep 17 17:42:00.393372 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:00.393330 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8l44" event={"ID":"3983db55-61ff-4e21-bfbb-b152cdd0eb9a","Type":"ContainerStarted","Data":"64277eee08a81629f5a469faaeb8b7246b3e909de3c8cd266b7f0c0d51d347ba"} Sep 17 17:42:01.399234 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:01.399190 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8l44" event={"ID":"3983db55-61ff-4e21-bfbb-b152cdd0eb9a","Type":"ContainerStarted","Data":"1c75451d1b2a4493fef59f802d6b9745ed2a0f253b0b647e60512ac64587a321"} Sep 17 17:42:01.422995 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:01.422942 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g8l44" podStartSLOduration=17.133654589 podStartE2EDuration="19.422923886s" totalImagesPullingTime="2.289269297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:41:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:01.421886812 +0000 UTC m=+65.571034232" watchObservedRunningTime="2026-09-17 17:42:01.422923886 +0000 UTC m=+65.572071306" Sep 17 17:42:03.697690 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.697643 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:42:03.697690 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.697701 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:42:03.700174 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.700150 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 17:42:03.700250 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.700183 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 17:42:03.710335 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.710301 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d3e728a9-fb95-4223-9920-37a2972a101c-original-pull-secret\") pod \"global-pull-secret-syncer-xs9vn\" (UID: \"d3e728a9-fb95-4223-9920-37a2972a101c\") " pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:42:03.710460 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.710349 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd1e8497-38f8-4cf2-b979-8cbadd3f4d38-metrics-certs\") pod \"network-metrics-daemon-xnrfz\" (UID: \"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38\") " pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:42:03.798225 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.798178 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:42:03.798399 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.798250 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:42:03.798399 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.798290 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:42:03.800892 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.800868 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/97f023e7-bed4-4556-8279-20e645880b88-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-f4hg9\" (UID: \"97f023e7-bed4-4556-8279-20e645880b88\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:42:03.801007 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.800869 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"image-registry-b8c4ff68f-n4x6c\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:42:03.801007 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.800873 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nn9j\" (UniqueName: \"kubernetes.io/projected/de127252-05c1-43c0-aba1-095c828851f9-kube-api-access-2nn9j\") pod \"network-check-target-wf9fj\" (UID: \"de127252-05c1-43c0-aba1-095c828851f9\") " pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:42:03.898955 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.898905 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:03.899136 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.898975 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:03.899136 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.899017 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:42:03.899656 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.899603 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9298287b-3797-48a7-995b-f613902a2a98-service-ca-bundle\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:03.901551 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.901524 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9298287b-3797-48a7-995b-f613902a2a98-metrics-certs\") pod \"router-default-7b6cb8c868-qgggf\" (UID: \"9298287b-3797-48a7-995b-f613902a2a98\") " pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:03.901672 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.901650 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6dc3cee-9579-4833-9632-1c03581ee3b7-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-ghqgb\" (UID: \"a6dc3cee-9579-4833-9632-1c03581ee3b7\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:42:03.937899 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.937867 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gwk7t\"" Sep 17 17:42:03.944356 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.944331 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hq5mj\"" Sep 17 17:42:03.945363 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.945344 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xnrfz" Sep 17 17:42:03.951044 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.951020 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xs9vn" Sep 17 17:42:03.952095 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.952073 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:42:04.001187 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:03.999781 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:42:04.004549 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.004418 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed64f380-f001-47b9-917e-990dd6ea473f-metrics-tls\") pod \"dns-default-sshtr\" (UID: \"ed64f380-f001-47b9-917e-990dd6ea473f\") " pod="openshift-dns/dns-default-sshtr" Sep 17 17:42:04.029087 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.018239 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mtf7c\"" Sep 17 17:42:04.029793 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.029760 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:42:04.034836 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.033234 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4hdqv\"" Sep 17 17:42:04.045218 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.044950 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" Sep 17 17:42:04.086410 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.086378 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7cfpq\"" Sep 17 17:42:04.094635 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.094590 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" Sep 17 17:42:04.101254 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.100770 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:42:04.105090 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.104880 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-p96kh\"" Sep 17 17:42:04.105090 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.105013 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e37515af-75c0-4ee7-8552-5212fc707e1f-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-sp2x7\" (UID: \"e37515af-75c0-4ee7-8552-5212fc707e1f\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:42:04.109648 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.109567 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:04.156128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.155877 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fglck\"" Sep 17 17:42:04.161905 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.161878 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sshtr" Sep 17 17:42:04.242360 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.242328 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xs9vn"] Sep 17 17:42:04.244057 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.243997 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xnrfz"] Sep 17 17:42:04.245944 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:04.245908 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1e8497_38f8_4cf2_b979_8cbadd3f4d38.slice/crio-524b317005b8d8f81e2f9fad36f227675bb56054413907cde82d15e3375937c1 WatchSource:0}: Error finding container 524b317005b8d8f81e2f9fad36f227675bb56054413907cde82d15e3375937c1: Status 404 returned error can't find the container with id 524b317005b8d8f81e2f9fad36f227675bb56054413907cde82d15e3375937c1 Sep 17 17:42:04.246567 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:04.246544 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e728a9_fb95_4223_9920_37a2972a101c.slice/crio-213c690274ef45dc5b3855d1cb8bd373b5a46ce7ef339bf391624ad5f7262660 WatchSource:0}: Error finding container 213c690274ef45dc5b3855d1cb8bd373b5a46ce7ef339bf391624ad5f7262660: Status 404 returned error can't find the container with id 213c690274ef45dc5b3855d1cb8bd373b5a46ce7ef339bf391624ad5f7262660 Sep 17 17:42:04.253308 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.253281 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wf9fj"] Sep 17 17:42:04.256932 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:04.256905 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde127252_05c1_43c0_aba1_095c828851f9.slice/crio-96f723b734fe03e8d461171d0ff8e32ba1d679b7b232c5206e27364014d1afd3 WatchSource:0}: Error finding container 96f723b734fe03e8d461171d0ff8e32ba1d679b7b232c5206e27364014d1afd3: Status 404 returned error can't find the container with id 96f723b734fe03e8d461171d0ff8e32ba1d679b7b232c5206e27364014d1afd3 Sep 17 17:42:04.284413 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.284388 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-n48lm\"" Sep 17 17:42:04.290697 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.290475 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" Sep 17 17:42:04.337462 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.337037 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9"] Sep 17 17:42:04.341677 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:04.341642 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f023e7_bed4_4556_8279_20e645880b88.slice/crio-883e7c95eb3660c735d2fdde48e12493eb5ed3b769cb40d3661426346f0827de WatchSource:0}: Error finding container 883e7c95eb3660c735d2fdde48e12493eb5ed3b769cb40d3661426346f0827de: Status 404 returned error can't find the container with id 883e7c95eb3660c735d2fdde48e12493eb5ed3b769cb40d3661426346f0827de Sep 17 17:42:04.343687 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.343599 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b8c4ff68f-n4x6c"] Sep 17 17:42:04.383915 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.383883 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb"] Sep 17 17:42:04.410033 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.410006 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xs9vn" event={"ID":"d3e728a9-fb95-4223-9920-37a2972a101c","Type":"ContainerStarted","Data":"213c690274ef45dc5b3855d1cb8bd373b5a46ce7ef339bf391624ad5f7262660"} Sep 17 17:42:04.411234 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.411180 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" event={"ID":"97f023e7-bed4-4556-8279-20e645880b88","Type":"ContainerStarted","Data":"883e7c95eb3660c735d2fdde48e12493eb5ed3b769cb40d3661426346f0827de"} Sep 17 17:42:04.412565 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.412543 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xnrfz" event={"ID":"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38","Type":"ContainerStarted","Data":"524b317005b8d8f81e2f9fad36f227675bb56054413907cde82d15e3375937c1"} Sep 17 17:42:04.414546 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.414520 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" event={"ID":"9d22ec60-8779-4a14-90f2-e8abff8d6b61","Type":"ContainerStarted","Data":"d905b2c627f1ebd4f94d837dcbe3ebf24e62edee2dde31bfdf881cc713ddcab1"} Sep 17 17:42:04.416119 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.416101 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wf9fj" event={"ID":"de127252-05c1-43c0-aba1-095c828851f9","Type":"ContainerStarted","Data":"fd8989ef5ba26968ebd061e0539ed37677b02d0ec32772b4c64bb1f0d1c448a6"} Sep 17 17:42:04.416210 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.416139 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wf9fj" event={"ID":"de127252-05c1-43c0-aba1-095c828851f9","Type":"ContainerStarted","Data":"96f723b734fe03e8d461171d0ff8e32ba1d679b7b232c5206e27364014d1afd3"} Sep 17 17:42:04.416273 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.416230 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:42:04.424647 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.424598 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b6cb8c868-qgggf"] Sep 17 17:42:04.428259 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:04.428231 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9298287b_3797_48a7_995b_f613902a2a98.slice/crio-f852d3447e828778aff7421c8d341d40c618f1c3746d55efdc2c8b01bad481a4 WatchSource:0}: Error finding container f852d3447e828778aff7421c8d341d40c618f1c3746d55efdc2c8b01bad481a4: Status 404 returned error can't find the container with id f852d3447e828778aff7421c8d341d40c618f1c3746d55efdc2c8b01bad481a4 Sep 17 17:42:04.463055 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.462996 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wf9fj" podStartSLOduration=66.462974623 podStartE2EDuration="1m6.462974623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:04.4476922 +0000 UTC m=+68.596839621" watchObservedRunningTime="2026-09-17 17:42:04.462974623 +0000 UTC m=+68.612122044" Sep 17 17:42:04.463895 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.463868 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sshtr"] Sep 17 17:42:04.468905 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:04.468866 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded64f380_f001_47b9_917e_990dd6ea473f.slice/crio-bc2e5566f0e9430d05dfcd8a073fe21181aea30bb18c0fb19bb0abac900d5f46 WatchSource:0}: Error finding container bc2e5566f0e9430d05dfcd8a073fe21181aea30bb18c0fb19bb0abac900d5f46: Status 404 returned error can't find the container with id bc2e5566f0e9430d05dfcd8a073fe21181aea30bb18c0fb19bb0abac900d5f46 Sep 17 17:42:04.583724 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:04.583687 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7"] Sep 17 17:42:04.587051 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:04.587025 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37515af_75c0_4ee7_8552_5212fc707e1f.slice/crio-ced05fae768bbcd667355da38c3047c38950384b999aa93734a345340a5473f3 WatchSource:0}: Error finding container ced05fae768bbcd667355da38c3047c38950384b999aa93734a345340a5473f3: Status 404 returned error can't find the container with id ced05fae768bbcd667355da38c3047c38950384b999aa93734a345340a5473f3 Sep 17 17:42:05.426160 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.426114 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" event={"ID":"9d22ec60-8779-4a14-90f2-e8abff8d6b61","Type":"ContainerStarted","Data":"754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb"} Sep 17 17:42:05.426657 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.426209 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:42:05.429312 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.429274 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" event={"ID":"a6dc3cee-9579-4833-9632-1c03581ee3b7","Type":"ContainerStarted","Data":"46acd2c6aca6a71199276ed8d74f7b50901d0fe1a81b18cf13e3ecdccd0ad97e"} Sep 17 17:42:05.431552 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.431496 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" event={"ID":"e37515af-75c0-4ee7-8552-5212fc707e1f","Type":"ContainerStarted","Data":"ced05fae768bbcd667355da38c3047c38950384b999aa93734a345340a5473f3"} Sep 17 17:42:05.435673 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.435586 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sshtr" event={"ID":"ed64f380-f001-47b9-917e-990dd6ea473f","Type":"ContainerStarted","Data":"bc2e5566f0e9430d05dfcd8a073fe21181aea30bb18c0fb19bb0abac900d5f46"} Sep 17 17:42:05.439806 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.439321 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" event={"ID":"9298287b-3797-48a7-995b-f613902a2a98","Type":"ContainerStarted","Data":"7eaefe4f32f79071c3b10452032fa50b5c26e800a6cfb164277096b38c10502d"} Sep 17 17:42:05.439806 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.439370 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" event={"ID":"9298287b-3797-48a7-995b-f613902a2a98","Type":"ContainerStarted","Data":"f852d3447e828778aff7421c8d341d40c618f1c3746d55efdc2c8b01bad481a4"} Sep 17 17:42:05.458848 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.458780 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" podStartSLOduration=67.458760303 podStartE2EDuration="1m7.458760303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:05.457008165 +0000 UTC m=+69.606155585" watchObservedRunningTime="2026-09-17 17:42:05.458760303 +0000 UTC m=+69.607907725" Sep 17 17:42:05.483734 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:05.483668 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" podStartSLOduration=46.483645108 podStartE2EDuration="46.483645108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:05.483334308 +0000 UTC m=+69.632481807" watchObservedRunningTime="2026-09-17 17:42:05.483645108 +0000 UTC m=+69.632792529" Sep 17 17:42:06.110355 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:06.110316 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:06.114003 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:06.113968 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:06.445554 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:06.445469 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xnrfz" event={"ID":"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38","Type":"ContainerStarted","Data":"2f5e65b161c4d804e85466eb76eed6001694b0b6ed17dced0c012b9e9a7f456c"} Sep 17 17:42:06.446029 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:06.445991 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:06.447544 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:06.447518 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7b6cb8c868-qgggf" Sep 17 17:42:07.184145 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.182867 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-856ks"] Sep 17 17:42:07.187590 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.187558 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-856ks" Sep 17 17:42:07.203519 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.203489 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-856ks"] Sep 17 17:42:07.206695 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.206668 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kqvlf\"" Sep 17 17:42:07.206852 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.206772 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 17 17:42:07.206852 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.206849 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 17 17:42:07.332079 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.331984 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88ts4\" (UniqueName: \"kubernetes.io/projected/8312e663-4022-4dde-9aab-2d9e52c2c8d8-kube-api-access-88ts4\") pod \"downloads-6594f4c75-856ks\" (UID: \"8312e663-4022-4dde-9aab-2d9e52c2c8d8\") " pod="openshift-console/downloads-6594f4c75-856ks" Sep 17 17:42:07.432743 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.432703 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88ts4\" (UniqueName: \"kubernetes.io/projected/8312e663-4022-4dde-9aab-2d9e52c2c8d8-kube-api-access-88ts4\") pod \"downloads-6594f4c75-856ks\" (UID: \"8312e663-4022-4dde-9aab-2d9e52c2c8d8\") " pod="openshift-console/downloads-6594f4c75-856ks" Sep 17 17:42:07.452265 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.452229 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88ts4\" (UniqueName: \"kubernetes.io/projected/8312e663-4022-4dde-9aab-2d9e52c2c8d8-kube-api-access-88ts4\") pod \"downloads-6594f4c75-856ks\" (UID: \"8312e663-4022-4dde-9aab-2d9e52c2c8d8\") " pod="openshift-console/downloads-6594f4c75-856ks" Sep 17 17:42:07.498902 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:07.498863 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-856ks" Sep 17 17:42:08.017379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.017283 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8f5fdfbbc-vz4vh"] Sep 17 17:42:08.020242 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.020219 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.026588 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:08.026549 2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-8f5fdfbbc-vz4vh\" is forbidden: User \"system:node:ip-10-0-138-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-138-30.ec2.internal' and this object" podUID="5f9fd21a-6efb-44f9-bd04-825605a47ac1" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.026980 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.026739 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 17 17:42:08.026980 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:08.026745 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-serving-cert\" is forbidden: User \"system:node:ip-10-0-138-30.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-138-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-serving-cert\"" type="*v1.Secret" Sep 17 17:42:08.026980 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.026804 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 17 17:42:08.026980 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.026910 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6kxlr\"" Sep 17 17:42:08.027213 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:08.026980 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-138-30.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-138-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" Sep 17 17:42:08.027213 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:08.027144 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"oauth-serving-cert\" is forbidden: User \"system:node:ip-10-0-138-30.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-138-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" type="*v1.ConfigMap" Sep 17 17:42:08.027213 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.027161 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 17 17:42:08.074055 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.074024 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8f5fdfbbc-vz4vh"] Sep 17 17:42:08.139007 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.138971 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-oauth-config\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.139219 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.139040 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.139219 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.139082 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-config\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.139219 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.139110 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwfvn\" (UniqueName: \"kubernetes.io/projected/5f9fd21a-6efb-44f9-bd04-825605a47ac1-kube-api-access-xwfvn\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.139219 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.139166 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.139432 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.139232 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-service-ca\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.139432 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.139260 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.240570 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.240530 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.240570 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.240573 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-config\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.240834 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.240598 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfvn\" (UniqueName: \"kubernetes.io/projected/5f9fd21a-6efb-44f9-bd04-825605a47ac1-kube-api-access-xwfvn\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.240834 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.240667 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.240834 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.240712 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-service-ca\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.240834 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.240752 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.240834 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.240777 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-oauth-config\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.241355 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.241328 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-config\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.241451 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.241402 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-service-ca\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.243745 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.243718 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-oauth-config\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:08.254491 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:08.254461 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwfvn\" (UniqueName: \"kubernetes.io/projected/5f9fd21a-6efb-44f9-bd04-825605a47ac1-kube-api-access-xwfvn\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:09.241056 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:09.240907 2586 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: failed to sync configmap cache: timed out waiting for the condition Sep 17 17:42:09.241056 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:09.240907 2586 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: failed to sync configmap cache: timed out waiting for the condition Sep 17 17:42:09.241056 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:09.240998 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert podName:5f9fd21a-6efb-44f9-bd04-825605a47ac1 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:09.740976573 +0000 UTC m=+73.890123988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert") pod "console-8f5fdfbbc-vz4vh" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1") : failed to sync configmap cache: timed out waiting for the condition Sep 17 17:42:09.241056 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:09.241016 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle podName:5f9fd21a-6efb-44f9-bd04-825605a47ac1 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:09.74100752 +0000 UTC m=+73.890154928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle") pod "console-8f5fdfbbc-vz4vh" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1") : failed to sync configmap cache: timed out waiting for the condition Sep 17 17:42:09.241056 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:09.241025 2586 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: failed to sync secret cache: timed out waiting for the condition Sep 17 17:42:09.241524 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:09.241112 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert podName:5f9fd21a-6efb-44f9-bd04-825605a47ac1 nodeName:}" failed. No retries permitted until 2026-09-17 17:42:09.741092938 +0000 UTC m=+73.890240364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert") pod "console-8f5fdfbbc-vz4vh" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1") : failed to sync secret cache: timed out waiting for the condition Sep 17 17:42:09.458780 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.457888 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 17 17:42:09.466591 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.466555 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" event={"ID":"a6dc3cee-9579-4833-9632-1c03581ee3b7","Type":"ContainerStarted","Data":"a82b7e87fae49ecbe0b87e09c8d14223acff0ab580374b7025528f2d1ab1e332"} Sep 17 17:42:09.466769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.466688 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" event={"ID":"a6dc3cee-9579-4833-9632-1c03581ee3b7","Type":"ContainerStarted","Data":"5dbab8ea37072e61bf778a8154c63c8e1305392f023799b3c6e884ab0ddab161"} Sep 17 17:42:09.469279 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.469237 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xs9vn" event={"ID":"d3e728a9-fb95-4223-9920-37a2972a101c","Type":"ContainerStarted","Data":"b66e3b344da3f0f5977f376faecc77472f68fbac3c5a84f3fba0ebf1bb577312"} Sep 17 17:42:09.473258 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.473207 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" event={"ID":"e37515af-75c0-4ee7-8552-5212fc707e1f","Type":"ContainerStarted","Data":"71211b6eef2996ec3fda0d134da3cd4553c47da2b39e08ef8b342ba570577092"} Sep 17 17:42:09.475202 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.475176 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sshtr" event={"ID":"ed64f380-f001-47b9-917e-990dd6ea473f","Type":"ContainerStarted","Data":"a29c904666196d32c754c45be7bccd38e46524bd66e3d373fcc35434d508cea4"} Sep 17 17:42:09.477585 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.477562 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" event={"ID":"97f023e7-bed4-4556-8279-20e645880b88","Type":"ContainerStarted","Data":"5aebe29faba7bdee22f3cbc3aebc809704efd8b7416b2ff345e2eb2c60410e40"} Sep 17 17:42:09.482642 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.482589 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xnrfz" event={"ID":"dd1e8497-38f8-4cf2-b979-8cbadd3f4d38","Type":"ContainerStarted","Data":"383cfa1118607d359f81bb30072df4079c88a6859bd7ffef39f0b636b168e33c"} Sep 17 17:42:09.485919 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.485848 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 17 17:42:09.498969 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.498926 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-ghqgb" podStartSLOduration=44.703753301 podStartE2EDuration="49.498908801s" totalImagesPullingTime="4.7951555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:09.497181528 +0000 UTC m=+73.646328963" watchObservedRunningTime="2026-09-17 17:42:09.498908801 +0000 UTC m=+73.648056220" Sep 17 17:42:09.517747 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.517712 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-856ks"] Sep 17 17:42:09.521169 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:09.521137 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8312e663_4022_4dde_9aab_2d9e52c2c8d8.slice/crio-2d6bffb0f391a80365bea2145de8817c496ddc214544bd41529bece78c4de2cc WatchSource:0}: Error finding container 2d6bffb0f391a80365bea2145de8817c496ddc214544bd41529bece78c4de2cc: Status 404 returned error can't find the container with id 2d6bffb0f391a80365bea2145de8817c496ddc214544bd41529bece78c4de2cc Sep 17 17:42:09.532674 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.532595 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-sp2x7" podStartSLOduration=35.934857302 podStartE2EDuration="40.532565004s" totalImagesPullingTime="4.597707702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:09.532038264 +0000 UTC m=+73.681185684" watchObservedRunningTime="2026-09-17 17:42:09.532565004 +0000 UTC m=+73.681712425" Sep 17 17:42:09.555834 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.555329 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xs9vn" podStartSLOduration=66.560628252 podStartE2EDuration="1m11.555310167s" totalImagesPullingTime="4.994681915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:09.553542522 +0000 UTC m=+73.702689943" watchObservedRunningTime="2026-09-17 17:42:09.555310167 +0000 UTC m=+73.704457588" Sep 17 17:42:09.616747 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.616671 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xnrfz" podStartSLOduration=70.400364735 podStartE2EDuration="1m11.616646143s" totalImagesPullingTime="1.216281408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:09.578124117 +0000 UTC m=+73.727271542" watchObservedRunningTime="2026-09-17 17:42:09.616646143 +0000 UTC m=+73.765793560" Sep 17 17:42:09.617224 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.617182 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-f4hg9" podStartSLOduration=45.771541235 podStartE2EDuration="50.617165878s" totalImagesPullingTime="4.845624643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:09.615300989 +0000 UTC m=+73.764448412" watchObservedRunningTime="2026-09-17 17:42:09.617165878 +0000 UTC m=+73.766313299" Sep 17 17:42:09.633678 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.633648 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 17 17:42:09.755149 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.755050 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:09.755149 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.755136 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:09.755364 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.755193 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:09.755989 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.755967 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:09.756057 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.755997 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:09.757740 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.757712 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert\") pod \"console-8f5fdfbbc-vz4vh\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:09.831840 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:09.831799 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:10.126945 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:10.126912 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8f5fdfbbc-vz4vh"] Sep 17 17:42:10.130231 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:10.130197 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9fd21a_6efb_44f9_bd04_825605a47ac1.slice/crio-a68f155e4c40c4283dac4539d9c9c6731bd1e47134579323ec65d9c8587e722c WatchSource:0}: Error finding container a68f155e4c40c4283dac4539d9c9c6731bd1e47134579323ec65d9c8587e722c: Status 404 returned error can't find the container with id a68f155e4c40c4283dac4539d9c9c6731bd1e47134579323ec65d9c8587e722c Sep 17 17:42:10.488810 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:10.488726 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f5fdfbbc-vz4vh" event={"ID":"5f9fd21a-6efb-44f9-bd04-825605a47ac1","Type":"ContainerStarted","Data":"a68f155e4c40c4283dac4539d9c9c6731bd1e47134579323ec65d9c8587e722c"} Sep 17 17:42:10.490791 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:10.490764 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sshtr" event={"ID":"ed64f380-f001-47b9-917e-990dd6ea473f","Type":"ContainerStarted","Data":"c6f9e07616e940ead1384845de657ed0e77a85e7bda3b630a5a696051b35dbac"} Sep 17 17:42:10.490924 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:10.490846 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sshtr" Sep 17 17:42:10.492279 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:10.492250 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-856ks" event={"ID":"8312e663-4022-4dde-9aab-2d9e52c2c8d8","Type":"ContainerStarted","Data":"2d6bffb0f391a80365bea2145de8817c496ddc214544bd41529bece78c4de2cc"} Sep 17 17:42:10.513750 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:10.513695 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sshtr" podStartSLOduration=34.774952569999996 podStartE2EDuration="39.513676022s" totalImagesPullingTime="4.738723452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:10.510275694 +0000 UTC m=+74.659423149" watchObservedRunningTime="2026-09-17 17:42:10.513676022 +0000 UTC m=+74.662823445" Sep 17 17:42:15.519316 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:15.519283 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f5fdfbbc-vz4vh" event={"ID":"5f9fd21a-6efb-44f9-bd04-825605a47ac1","Type":"ContainerStarted","Data":"f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf"} Sep 17 17:42:15.539961 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:15.539903 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8f5fdfbbc-vz4vh" podStartSLOduration=3.760073825 podStartE2EDuration="8.539881741s" totalImagesPullingTime="4.779807916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:15.539838142 +0000 UTC m=+79.688985561" watchObservedRunningTime="2026-09-17 17:42:15.539881741 +0000 UTC m=+79.689029154" Sep 17 17:42:17.031795 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.031755 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5"] Sep 17 17:42:17.034379 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.034352 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.036847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.036646 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 17 17:42:17.036847 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.036812 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 17 17:42:17.037391 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.037369 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 17:42:17.037642 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.037377 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vt2lg\"" Sep 17 17:42:17.050079 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.050050 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5"] Sep 17 17:42:17.125730 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.125670 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/46659d9e-fe98-4c53-8d6d-227825b7c492-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.125927 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.125745 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46659d9e-fe98-4c53-8d6d-227825b7c492-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.125927 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.125780 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46659d9e-fe98-4c53-8d6d-227825b7c492-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.125927 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.125799 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vr7t\" (UniqueName: \"kubernetes.io/projected/46659d9e-fe98-4c53-8d6d-227825b7c492-kube-api-access-4vr7t\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.227026 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.226982 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/46659d9e-fe98-4c53-8d6d-227825b7c492-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.227205 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.227046 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46659d9e-fe98-4c53-8d6d-227825b7c492-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.227205 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.227093 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46659d9e-fe98-4c53-8d6d-227825b7c492-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.227205 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.227124 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vr7t\" (UniqueName: \"kubernetes.io/projected/46659d9e-fe98-4c53-8d6d-227825b7c492-kube-api-access-4vr7t\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.228155 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.228115 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46659d9e-fe98-4c53-8d6d-227825b7c492-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.230142 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.230118 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46659d9e-fe98-4c53-8d6d-227825b7c492-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.230348 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.230323 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/46659d9e-fe98-4c53-8d6d-227825b7c492-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.244396 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.244359 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vr7t\" (UniqueName: \"kubernetes.io/projected/46659d9e-fe98-4c53-8d6d-227825b7c492-kube-api-access-4vr7t\") pod \"prometheus-operator-7bd5d56d6-zxjq5\" (UID: \"46659d9e-fe98-4c53-8d6d-227825b7c492\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.345276 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.345193 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" Sep 17 17:42:17.642595 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:17.642555 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5"] Sep 17 17:42:17.648119 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:17.648088 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46659d9e_fe98_4c53_8d6d_227825b7c492.slice/crio-c46f8fb269a07f3c96650cf1ebc7473bf171b39e926072f3831a3c67b5f2fff8 WatchSource:0}: Error finding container c46f8fb269a07f3c96650cf1ebc7473bf171b39e926072f3831a3c67b5f2fff8: Status 404 returned error can't find the container with id c46f8fb269a07f3c96650cf1ebc7473bf171b39e926072f3831a3c67b5f2fff8 Sep 17 17:42:18.534710 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:18.534657 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" event={"ID":"46659d9e-fe98-4c53-8d6d-227825b7c492","Type":"ContainerStarted","Data":"c46f8fb269a07f3c96650cf1ebc7473bf171b39e926072f3831a3c67b5f2fff8"} Sep 17 17:42:19.541719 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:19.541680 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" event={"ID":"46659d9e-fe98-4c53-8d6d-227825b7c492","Type":"ContainerStarted","Data":"07cdbbb59971ebee64f03a8a57a49b1406248dab63ac12a06c7d68313f303287"} Sep 17 17:42:19.542168 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:19.541786 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" event={"ID":"46659d9e-fe98-4c53-8d6d-227825b7c492","Type":"ContainerStarted","Data":"4cbfee74f553d4e4661da752c0d6bc002442c9178f9538dae66b76ff23afee8e"} Sep 17 17:42:19.574476 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:19.574407 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-zxjq5" podStartSLOduration=1.320596368 podStartE2EDuration="2.574387261s" totalImagesPullingTime="1.253790893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:19.573090351 +0000 UTC m=+83.722237776" watchObservedRunningTime="2026-09-17 17:42:19.574387261 +0000 UTC m=+83.723534681" Sep 17 17:42:19.832263 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:19.832172 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:19.832263 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:19.832257 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:19.838247 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:19.838220 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:20.502807 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:20.502773 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sshtr" Sep 17 17:42:20.552295 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:20.552262 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:42:21.605891 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.605853 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5pfsf"] Sep 17 17:42:21.608911 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.608879 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.611224 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.611193 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 17:42:21.611357 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.611269 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 17:42:21.611484 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.611466 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 17:42:21.611722 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.611700 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ql85w\"" Sep 17 17:42:21.776109 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776074 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-sys\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.776109 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776112 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a11b0c2-a065-458f-876c-15861ee77e8b-metrics-client-ca\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.776339 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776145 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-tls\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.776339 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776229 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.776339 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776317 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-textfile\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.776339 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776341 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxpmr\" (UniqueName: \"kubernetes.io/projected/6a11b0c2-a065-458f-876c-15861ee77e8b-kube-api-access-bxpmr\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.776508 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776366 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-wtmp\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.776508 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776387 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-root\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.776508 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.776475 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877533 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877444 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-wtmp\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877533 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877497 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-root\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877536 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877569 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-sys\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877592 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a11b0c2-a065-458f-876c-15861ee77e8b-metrics-client-ca\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877640 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-tls\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877676 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877683 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-wtmp\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877728 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-sys\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877693 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a11b0c2-a065-458f-876c-15861ee77e8b-root\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.877785 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877785 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-textfile\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.878231 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.877814 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpmr\" (UniqueName: \"kubernetes.io/projected/6a11b0c2-a065-458f-876c-15861ee77e8b-kube-api-access-bxpmr\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.878231 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.878205 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-textfile\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.878332 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.878302 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-accelerators-collector-config\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.878332 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.878316 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a11b0c2-a065-458f-876c-15861ee77e8b-metrics-client-ca\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.880688 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.880664 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.882030 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.882007 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a11b0c2-a065-458f-876c-15861ee77e8b-node-exporter-tls\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.910006 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.909963 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxpmr\" (UniqueName: \"kubernetes.io/projected/6a11b0c2-a065-458f-876c-15861ee77e8b-kube-api-access-bxpmr\") pod \"node-exporter-5pfsf\" (UID: \"6a11b0c2-a065-458f-876c-15861ee77e8b\") " pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:21.919342 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:21.919311 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5pfsf" Sep 17 17:42:26.451170 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:26.451136 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:42:26.751588 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:26.751556 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a11b0c2_a065_458f_876c_15861ee77e8b.slice/crio-d2772275319d2673caefb22ec466ebfa091f1313797d1198cb0953a76a2fca8a WatchSource:0}: Error finding container d2772275319d2673caefb22ec466ebfa091f1313797d1198cb0953a76a2fca8a: Status 404 returned error can't find the container with id d2772275319d2673caefb22ec466ebfa091f1313797d1198cb0953a76a2fca8a Sep 17 17:42:27.582516 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.582474 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-856ks" event={"ID":"8312e663-4022-4dde-9aab-2d9e52c2c8d8","Type":"ContainerStarted","Data":"6ca4fed1a66dbcebefdde4795095e40f82480c90f2bac67042cbee3bb9cf7712"} Sep 17 17:42:27.583300 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.583267 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-856ks" Sep 17 17:42:27.585779 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.585717 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5pfsf" event={"ID":"6a11b0c2-a065-458f-876c-15861ee77e8b","Type":"ContainerStarted","Data":"d2772275319d2673caefb22ec466ebfa091f1313797d1198cb0953a76a2fca8a"} Sep 17 17:42:27.593821 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.593755 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-856ks" Sep 17 17:42:27.603965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.602425 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-856ks" podStartSLOduration=3.087298881 podStartE2EDuration="20.602406522s" totalImagesPullingTime="17.515107641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:27.601507598 +0000 UTC m=+91.750655019" watchObservedRunningTime="2026-09-17 17:42:27.602406522 +0000 UTC m=+91.751554002" Sep 17 17:42:27.821675 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.820191 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 17:42:27.827523 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.827488 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.830918 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.830159 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 17 17:42:27.830918 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.830411 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4cj5g1ppj316o\"" Sep 17 17:42:27.830918 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.830837 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 17 17:42:27.831168 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.831148 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 17 17:42:27.831388 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.831371 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 17 17:42:27.831584 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.831565 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 17 17:42:27.833149 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.831780 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 17 17:42:27.833149 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.832753 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 17 17:42:27.833149 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.832825 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 17 17:42:27.833480 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.833455 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 17 17:42:27.835753 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.833651 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 17 17:42:27.835753 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.833957 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 17 17:42:27.835753 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.834120 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nhcfl\"" Sep 17 17:42:27.835753 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.834189 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 17 17:42:27.835753 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.835606 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 17 17:42:27.837166 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.836922 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 17 17:42:27.839945 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.839925 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 17 17:42:27.845763 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.845403 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 17:42:27.926220 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926187 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926381 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926231 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926381 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926271 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926381 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926376 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926570 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926403 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926570 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926458 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926570 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926539 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926755 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926566 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926755 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926687 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53a68631-6c30-46f8-b5c2-009d00ab0cda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926755 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926732 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-config\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926904 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926761 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53a68631-6c30-46f8-b5c2-009d00ab0cda-config-out\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926904 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926789 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926904 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926816 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926904 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926841 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-web-config\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.926904 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926871 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.927146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926911 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.927146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926958 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.927146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.926991 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.927146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.927017 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:27.927146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:27.927060 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t9fd\" (UniqueName: \"kubernetes.io/projected/53a68631-6c30-46f8-b5c2-009d00ab0cda-kube-api-access-6t9fd\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.027686 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027640 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t9fd\" (UniqueName: \"kubernetes.io/projected/53a68631-6c30-46f8-b5c2-009d00ab0cda-kube-api-access-6t9fd\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.027874 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027699 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.027874 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027741 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.027874 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027791 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.027874 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027825 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.027874 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027849 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.028146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027874 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.028146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027921 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.028146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027949 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.028146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.027980 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53a68631-6c30-46f8-b5c2-009d00ab0cda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.028146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.028023 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-config\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.028146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.028049 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53a68631-6c30-46f8-b5c2-009d00ab0cda-config-out\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.028146 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.028077 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.029520 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.029029 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.029696 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.029671 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.030352 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.030067 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.032541 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.032640 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.032670 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-web-config\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.032704 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.032748 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.032801 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.032833 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.032859 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.033494 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.033821 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.035281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.034449 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.036491 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.036113 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.039015 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.038991 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-web-config\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.039767 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.039693 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.040669 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.040086 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53a68631-6c30-46f8-b5c2-009d00ab0cda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.040669 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.040540 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53a68631-6c30-46f8-b5c2-009d00ab0cda-config-out\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.040846 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.040818 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53a68631-6c30-46f8-b5c2-009d00ab0cda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.042158 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.042094 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-config\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.043462 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.043410 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.043827 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.043785 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t9fd\" (UniqueName: \"kubernetes.io/projected/53a68631-6c30-46f8-b5c2-009d00ab0cda-kube-api-access-6t9fd\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.043827 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.043818 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.043961 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.043946 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.044589 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.044569 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.044758 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.044671 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53a68631-6c30-46f8-b5c2-009d00ab0cda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53a68631-6c30-46f8-b5c2-009d00ab0cda\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.152030 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.146379 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:28.530737 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:42:28.530692 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a68631_6c30_46f8_b5c2_009d00ab0cda.slice/crio-890e12a1b5ec291949fbcdd4197ce90d7bb1fa9c1286f07840feec80f4bae02e WatchSource:0}: Error finding container 890e12a1b5ec291949fbcdd4197ce90d7bb1fa9c1286f07840feec80f4bae02e: Status 404 returned error can't find the container with id 890e12a1b5ec291949fbcdd4197ce90d7bb1fa9c1286f07840feec80f4bae02e Sep 17 17:42:28.531138 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.531110 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 17 17:42:28.592480 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.592440 2586 generic.go:356] "Generic (PLEG): container finished" podID="6a11b0c2-a065-458f-876c-15861ee77e8b" containerID="8f6a8e8eb2d1b5488ef44e2d69a571619e279a507f10ed570857a59d76f5e7ac" exitCode=0 Sep 17 17:42:28.592982 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.592545 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5pfsf" event={"ID":"6a11b0c2-a065-458f-876c-15861ee77e8b","Type":"ContainerDied","Data":"8f6a8e8eb2d1b5488ef44e2d69a571619e279a507f10ed570857a59d76f5e7ac"} Sep 17 17:42:28.594848 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:28.594809 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53a68631-6c30-46f8-b5c2-009d00ab0cda","Type":"ContainerStarted","Data":"890e12a1b5ec291949fbcdd4197ce90d7bb1fa9c1286f07840feec80f4bae02e"} Sep 17 17:42:29.603804 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:29.603754 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5pfsf" event={"ID":"6a11b0c2-a065-458f-876c-15861ee77e8b","Type":"ContainerStarted","Data":"b42a9f8405ac095bb659cb475d31dc0aeb138583cb66340a3ddf991b0e381703"} Sep 17 17:42:29.604361 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:29.603840 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5pfsf" event={"ID":"6a11b0c2-a065-458f-876c-15861ee77e8b","Type":"ContainerStarted","Data":"3ca6218489defab4d6287c12627fb1af063e7c55789406c405d175ba717a803b"} Sep 17 17:42:29.660462 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:29.660405 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5pfsf" podStartSLOduration=7.402735645 podStartE2EDuration="8.660389787s" totalImagesPullingTime="1.15312049s" totalInitContainerRuntime="104.533652ms" isStatefulPod=true podCreationTimestamp="2026-09-17 17:42:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:29.656322493 +0000 UTC m=+93.805469925" watchObservedRunningTime="2026-09-17 17:42:29.660389787 +0000 UTC m=+93.809537206" Sep 17 17:42:30.611072 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:30.611030 2586 generic.go:356] "Generic (PLEG): container finished" podID="53a68631-6c30-46f8-b5c2-009d00ab0cda" containerID="05d8d2595b6e8ec40cc8af3a8de58e768a4a64fa047b433060056ab513da3da4" exitCode=0 Sep 17 17:42:30.611543 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:30.611131 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53a68631-6c30-46f8-b5c2-009d00ab0cda","Type":"ContainerDied","Data":"05d8d2595b6e8ec40cc8af3a8de58e768a4a64fa047b433060056ab513da3da4"} Sep 17 17:42:30.914694 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:30.914600 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b8c4ff68f-n4x6c"] Sep 17 17:42:35.443295 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:35.443258 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wf9fj" Sep 17 17:42:35.642140 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:35.642100 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53a68631-6c30-46f8-b5c2-009d00ab0cda","Type":"ContainerStarted","Data":"35aab3a4791a7ce9243905c544db7abf291c26aee2f4cd1a35204e4d71015281"} Sep 17 17:42:35.642443 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:35.642411 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53a68631-6c30-46f8-b5c2-009d00ab0cda","Type":"ContainerStarted","Data":"bead899b9ca3421701d56693bb944dc3eb55538f150d3f7ce37b23c9bc642440"} Sep 17 17:42:37.023749 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:37.023711 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8f5fdfbbc-vz4vh"] Sep 17 17:42:38.658656 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:38.658603 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53a68631-6c30-46f8-b5c2-009d00ab0cda","Type":"ContainerStarted","Data":"3d74e85d060932cd687a6ffe3bd821bcee7e5058c1c19444a1907ab379a13d03"} Sep 17 17:42:38.659078 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:38.658685 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53a68631-6c30-46f8-b5c2-009d00ab0cda","Type":"ContainerStarted","Data":"30b4745a0964f1a561be36b9eabe24437ed357c4aa7e236fd83a53dc9b517792"} Sep 17 17:42:38.659078 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:38.658728 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53a68631-6c30-46f8-b5c2-009d00ab0cda","Type":"ContainerStarted","Data":"ceba7998d89bf427082313311e810f430de5aae0e6be9929ce59c6c6b67f948a"} Sep 17 17:42:38.659078 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:38.658756 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53a68631-6c30-46f8-b5c2-009d00ab0cda","Type":"ContainerStarted","Data":"a13ec13909471d6032b21cb8b3bf88094ef679cf871559df936055b2e1b81b46"} Sep 17 17:42:38.688306 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:38.688246 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.177540369 podStartE2EDuration="11.68822374s" totalImagesPullingTime="8.495468119s" totalInitContainerRuntime="15.215252ms" isStatefulPod=false podCreationTimestamp="2026-09-17 17:42:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:42:38.686967724 +0000 UTC m=+102.836115156" watchObservedRunningTime="2026-09-17 17:42:38.68822374 +0000 UTC m=+102.837371160" Sep 17 17:42:43.147197 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:43.147155 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:42:55.938491 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:55.938426 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" podUID="9d22ec60-8779-4a14-90f2-e8abff8d6b61" containerName="registry" containerID="cri-o://754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb" gracePeriod=30 Sep 17 17:42:56.186342 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.186314 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:42:56.305275 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.305188 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-bound-sa-token\") pod \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " Sep 17 17:42:56.305275 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.305254 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-758rz\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-kube-api-access-758rz\") pod \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " Sep 17 17:42:56.305495 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.305302 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-image-registry-private-configuration\") pod \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " Sep 17 17:42:56.305495 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.305339 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") pod \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " Sep 17 17:42:56.305495 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.305377 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-installation-pull-secrets\") pod \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " Sep 17 17:42:56.305495 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.305405 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-trusted-ca\") pod \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " Sep 17 17:42:56.305495 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.305443 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d22ec60-8779-4a14-90f2-e8abff8d6b61-ca-trust-extracted\") pod \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " Sep 17 17:42:56.305495 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.305470 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-certificates\") pod \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\" (UID: \"9d22ec60-8779-4a14-90f2-e8abff8d6b61\") " Sep 17 17:42:56.306280 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.306201 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9d22ec60-8779-4a14-90f2-e8abff8d6b61" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:42:56.306408 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.306273 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9d22ec60-8779-4a14-90f2-e8abff8d6b61" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:42:56.308115 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.308088 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9d22ec60-8779-4a14-90f2-e8abff8d6b61" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:42:56.308288 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.308255 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-kube-api-access-758rz" (OuterVolumeSpecName: "kube-api-access-758rz") pod "9d22ec60-8779-4a14-90f2-e8abff8d6b61" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61"). InnerVolumeSpecName "kube-api-access-758rz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:42:56.308414 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.308246 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9d22ec60-8779-4a14-90f2-e8abff8d6b61" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:42:56.308476 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.308410 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9d22ec60-8779-4a14-90f2-e8abff8d6b61" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:42:56.308720 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.308693 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9d22ec60-8779-4a14-90f2-e8abff8d6b61" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:42:56.314271 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.314229 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d22ec60-8779-4a14-90f2-e8abff8d6b61-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9d22ec60-8779-4a14-90f2-e8abff8d6b61" (UID: "9d22ec60-8779-4a14-90f2-e8abff8d6b61"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 17:42:56.406526 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.406486 2586 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-bound-sa-token\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:42:56.406526 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.406519 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-758rz\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-kube-api-access-758rz\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:42:56.406526 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.406534 2586 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-image-registry-private-configuration\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:42:56.406802 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.406549 2586 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-tls\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:42:56.406802 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.406562 2586 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9d22ec60-8779-4a14-90f2-e8abff8d6b61-installation-pull-secrets\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:42:56.406802 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.406574 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-trusted-ca\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:42:56.406802 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.406587 2586 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9d22ec60-8779-4a14-90f2-e8abff8d6b61-ca-trust-extracted\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:42:56.406802 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.406598 2586 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9d22ec60-8779-4a14-90f2-e8abff8d6b61-registry-certificates\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:42:56.741334 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.741297 2586 generic.go:356] "Generic (PLEG): container finished" podID="9d22ec60-8779-4a14-90f2-e8abff8d6b61" containerID="754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb" exitCode=0 Sep 17 17:42:56.741518 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.741345 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" event={"ID":"9d22ec60-8779-4a14-90f2-e8abff8d6b61","Type":"ContainerDied","Data":"754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb"} Sep 17 17:42:56.741518 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.741354 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" Sep 17 17:42:56.741518 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.741409 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b8c4ff68f-n4x6c" event={"ID":"9d22ec60-8779-4a14-90f2-e8abff8d6b61","Type":"ContainerDied","Data":"d905b2c627f1ebd4f94d837dcbe3ebf24e62edee2dde31bfdf881cc713ddcab1"} Sep 17 17:42:56.741518 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.741428 2586 scope.go:117] "RemoveContainer" containerID="754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb" Sep 17 17:42:56.750961 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.750942 2586 scope.go:117] "RemoveContainer" containerID="754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb" Sep 17 17:42:56.751317 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:42:56.751296 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb\": container with ID starting with 754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb not found: ID does not exist" containerID="754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb" Sep 17 17:42:56.751363 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.751326 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb"} err="failed to get container status \"754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb\": rpc error: code = NotFound desc = could not find container \"754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb\": container with ID starting with 754dbf419e0ec7f01c6adc88c724639f3926416535da45cee5cb4c31179b9ddb not found: ID does not exist" Sep 17 17:42:56.762730 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.762697 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b8c4ff68f-n4x6c"] Sep 17 17:42:56.766322 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:56.766296 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b8c4ff68f-n4x6c"] Sep 17 17:42:58.038065 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:42:58.038029 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d22ec60-8779-4a14-90f2-e8abff8d6b61" path="/var/lib/kubelet/pods/9d22ec60-8779-4a14-90f2-e8abff8d6b61/volumes" Sep 17 17:43:02.047182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.047135 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8f5fdfbbc-vz4vh" podUID="5f9fd21a-6efb-44f9-bd04-825605a47ac1" containerName="console" containerID="cri-o://f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf" gracePeriod=15 Sep 17 17:43:02.331481 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.331446 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:43:02.456941 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.456840 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert\") pod \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " Sep 17 17:43:02.456941 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.456897 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-config\") pod \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " Sep 17 17:43:02.456941 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.456931 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-oauth-config\") pod \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " Sep 17 17:43:02.457212 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.456964 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-service-ca\") pod \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " Sep 17 17:43:02.457212 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.457039 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwfvn\" (UniqueName: \"kubernetes.io/projected/5f9fd21a-6efb-44f9-bd04-825605a47ac1-kube-api-access-xwfvn\") pod \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " Sep 17 17:43:02.457212 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.457072 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle\") pod \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " Sep 17 17:43:02.457212 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.457125 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert\") pod \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\" (UID: \"5f9fd21a-6efb-44f9-bd04-825605a47ac1\") " Sep 17 17:43:02.457387 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.457243 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5f9fd21a-6efb-44f9-bd04-825605a47ac1" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:43:02.457473 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.457445 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-config" (OuterVolumeSpecName: "console-config") pod "5f9fd21a-6efb-44f9-bd04-825605a47ac1" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:43:02.457516 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.457490 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5f9fd21a-6efb-44f9-bd04-825605a47ac1" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:43:02.457556 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.457511 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-service-ca" (OuterVolumeSpecName: "service-ca") pod "5f9fd21a-6efb-44f9-bd04-825605a47ac1" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:43:02.459637 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.459588 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5f9fd21a-6efb-44f9-bd04-825605a47ac1" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:43:02.459759 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.459736 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5f9fd21a-6efb-44f9-bd04-825605a47ac1" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:43:02.459835 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.459812 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f9fd21a-6efb-44f9-bd04-825605a47ac1-kube-api-access-xwfvn" (OuterVolumeSpecName: "kube-api-access-xwfvn") pod "5f9fd21a-6efb-44f9-bd04-825605a47ac1" (UID: "5f9fd21a-6efb-44f9-bd04-825605a47ac1"). InnerVolumeSpecName "kube-api-access-xwfvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:43:02.558056 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.558013 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwfvn\" (UniqueName: \"kubernetes.io/projected/5f9fd21a-6efb-44f9-bd04-825605a47ac1-kube-api-access-xwfvn\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:43:02.558056 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.558046 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-trusted-ca-bundle\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:43:02.558056 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.558055 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-serving-cert\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:43:02.558056 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.558065 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-oauth-serving-cert\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:43:02.558324 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.558073 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-config\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:43:02.558324 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.558082 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f9fd21a-6efb-44f9-bd04-825605a47ac1-console-oauth-config\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:43:02.558324 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.558090 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f9fd21a-6efb-44f9-bd04-825605a47ac1-service-ca\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:43:02.771178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.771087 2586 generic.go:356] "Generic (PLEG): container finished" podID="ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8" containerID="b2087acd8f48aa23621f45e58f422a581da5ff350f5bcc97b7f09b0c83ce1bca" exitCode=0 Sep 17 17:43:02.771346 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.771165 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" event={"ID":"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8","Type":"ContainerDied","Data":"b2087acd8f48aa23621f45e58f422a581da5ff350f5bcc97b7f09b0c83ce1bca"} Sep 17 17:43:02.771540 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.771521 2586 scope.go:117] "RemoveContainer" containerID="b2087acd8f48aa23621f45e58f422a581da5ff350f5bcc97b7f09b0c83ce1bca" Sep 17 17:43:02.772546 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.772518 2586 generic.go:356] "Generic (PLEG): container finished" podID="5f9fd21a-6efb-44f9-bd04-825605a47ac1" containerID="f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf" exitCode=0 Sep 17 17:43:02.772695 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.772557 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f5fdfbbc-vz4vh" event={"ID":"5f9fd21a-6efb-44f9-bd04-825605a47ac1","Type":"ContainerDied","Data":"f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf"} Sep 17 17:43:02.772695 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.772565 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f5fdfbbc-vz4vh" Sep 17 17:43:02.772695 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.772601 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f5fdfbbc-vz4vh" event={"ID":"5f9fd21a-6efb-44f9-bd04-825605a47ac1","Type":"ContainerDied","Data":"a68f155e4c40c4283dac4539d9c9c6731bd1e47134579323ec65d9c8587e722c"} Sep 17 17:43:02.772695 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.772658 2586 scope.go:117] "RemoveContainer" containerID="f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf" Sep 17 17:43:02.781388 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.781369 2586 scope.go:117] "RemoveContainer" containerID="f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf" Sep 17 17:43:02.781731 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:43:02.781708 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf\": container with ID starting with f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf not found: ID does not exist" containerID="f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf" Sep 17 17:43:02.781789 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.781740 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf"} err="failed to get container status \"f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf\": rpc error: code = NotFound desc = could not find container \"f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf\": container with ID starting with f3c6bb525a70bbe62050f1d7a4fcbb16e5c5d432cd5ca92f973e1c7bfe0caaaf not found: ID does not exist" Sep 17 17:43:02.806515 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.806470 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8f5fdfbbc-vz4vh"] Sep 17 17:43:02.814538 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:02.814455 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8f5fdfbbc-vz4vh"] Sep 17 17:43:03.778796 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:03.778747 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-fslx6" event={"ID":"ea9c4c8f-28b5-4f82-a0f0-cf1c7dfe1ff8","Type":"ContainerStarted","Data":"8843988caae60bb12402efb018bbc137e0b3e393aa0e5449b278d7e1ac775d8c"} Sep 17 17:43:04.038309 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:04.038208 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f9fd21a-6efb-44f9-bd04-825605a47ac1" path="/var/lib/kubelet/pods/5f9fd21a-6efb-44f9-bd04-825605a47ac1/volumes" Sep 17 17:43:06.793667 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:06.793608 2586 generic.go:356] "Generic (PLEG): container finished" podID="778a9f61-f31c-4157-9e8c-69c40f0dd7a3" containerID="e1de97cbe28a4648a8367e813220624edf1372efb87362250de6dcbf17ce22d9" exitCode=0 Sep 17 17:43:06.794049 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:06.793721 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" event={"ID":"778a9f61-f31c-4157-9e8c-69c40f0dd7a3","Type":"ContainerDied","Data":"e1de97cbe28a4648a8367e813220624edf1372efb87362250de6dcbf17ce22d9"} Sep 17 17:43:06.794095 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:06.794064 2586 scope.go:117] "RemoveContainer" containerID="e1de97cbe28a4648a8367e813220624edf1372efb87362250de6dcbf17ce22d9" Sep 17 17:43:07.799431 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:07.799397 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-nhxzc" event={"ID":"778a9f61-f31c-4157-9e8c-69c40f0dd7a3","Type":"ContainerStarted","Data":"a1365817ed1c3c4c21d93fcaf2bf46ba5308e069b0115546614c552baf3203f6"} Sep 17 17:43:11.815940 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:11.815904 2586 generic.go:356] "Generic (PLEG): container finished" podID="19467e99-0315-4947-9934-80d423a3e33d" containerID="20df0b9f873df472ce701d15ee418feda3e5bb85e2bbbd50b96d6304a05df483" exitCode=0 Sep 17 17:43:11.816509 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:11.815984 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" event={"ID":"19467e99-0315-4947-9934-80d423a3e33d","Type":"ContainerDied","Data":"20df0b9f873df472ce701d15ee418feda3e5bb85e2bbbd50b96d6304a05df483"} Sep 17 17:43:11.816509 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:11.816462 2586 scope.go:117] "RemoveContainer" containerID="20df0b9f873df472ce701d15ee418feda3e5bb85e2bbbd50b96d6304a05df483" Sep 17 17:43:12.822081 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:12.822047 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-lzhcp" event={"ID":"19467e99-0315-4947-9934-80d423a3e33d","Type":"ContainerStarted","Data":"efa9cf45649e035a3d021aad93748177fc9680efb7258263e45b76270ca75648"} Sep 17 17:43:28.146836 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:28.146801 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:43:28.162408 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:28.162381 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:43:28.903412 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:43:28.903380 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 17 17:45:57.927278 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:45:57.927248 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:45:57.933662 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:45:57.933637 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:45:57.939529 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:45:57.939505 2586 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 17:48:43.198401 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.198360 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-mvn7n"] Sep 17 17:48:43.198910 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.198662 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f9fd21a-6efb-44f9-bd04-825605a47ac1" containerName="console" Sep 17 17:48:43.198910 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.198674 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f9fd21a-6efb-44f9-bd04-825605a47ac1" containerName="console" Sep 17 17:48:43.198910 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.198691 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d22ec60-8779-4a14-90f2-e8abff8d6b61" containerName="registry" Sep 17 17:48:43.198910 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.198696 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d22ec60-8779-4a14-90f2-e8abff8d6b61" containerName="registry" Sep 17 17:48:43.198910 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.198749 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f9fd21a-6efb-44f9-bd04-825605a47ac1" containerName="console" Sep 17 17:48:43.198910 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.198758 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d22ec60-8779-4a14-90f2-e8abff8d6b61" containerName="registry" Sep 17 17:48:43.200638 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.200608 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-mvn7n" Sep 17 17:48:43.203779 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.203758 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\"" Sep 17 17:48:43.204253 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.204224 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Sep 17 17:48:43.204253 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.204247 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Sep 17 17:48:43.204406 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.204257 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-zlkc6\"" Sep 17 17:48:43.205732 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.205708 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Sep 17 17:48:43.213288 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.213268 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-mvn7n"] Sep 17 17:48:43.266452 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.266426 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjzfm\" (UniqueName: \"kubernetes.io/projected/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-kube-api-access-bjzfm\") pod \"s3-init-mvn7n\" (UID: \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\") " pod="kserve/s3-init-mvn7n" Sep 17 17:48:43.266570 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.266474 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-model-cache\") pod \"s3-init-mvn7n\" (UID: \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\") " pod="kserve/s3-init-mvn7n" Sep 17 17:48:43.367366 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.367335 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-model-cache\") pod \"s3-init-mvn7n\" (UID: \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\") " pod="kserve/s3-init-mvn7n" Sep 17 17:48:43.367499 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.367424 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjzfm\" (UniqueName: \"kubernetes.io/projected/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-kube-api-access-bjzfm\") pod \"s3-init-mvn7n\" (UID: \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\") " pod="kserve/s3-init-mvn7n" Sep 17 17:48:43.367827 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.367800 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-model-cache\") pod \"s3-init-mvn7n\" (UID: \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\") " pod="kserve/s3-init-mvn7n" Sep 17 17:48:43.376170 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.376149 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjzfm\" (UniqueName: \"kubernetes.io/projected/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-kube-api-access-bjzfm\") pod \"s3-init-mvn7n\" (UID: \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\") " pod="kserve/s3-init-mvn7n" Sep 17 17:48:43.521209 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.521142 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-mvn7n" Sep 17 17:48:43.792374 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.792290 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-mvn7n"] Sep 17 17:48:43.794956 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:48:43.794926 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ec2c2ce_f60d_4dc4_8fe5_286fd989b5ba.slice/crio-b2215a660108f8485b269f5927653d4e323a28539ea045ee81023441f2580345 WatchSource:0}: Error finding container b2215a660108f8485b269f5927653d4e323a28539ea045ee81023441f2580345: Status 404 returned error can't find the container with id b2215a660108f8485b269f5927653d4e323a28539ea045ee81023441f2580345 Sep 17 17:48:43.796856 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:43.796842 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 17:48:44.110647 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:44.110587 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mvn7n" event={"ID":"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba","Type":"ContainerStarted","Data":"b2215a660108f8485b269f5927653d4e323a28539ea045ee81023441f2580345"} Sep 17 17:48:48.129223 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:48.129187 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mvn7n" event={"ID":"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba","Type":"ContainerStarted","Data":"a99651648b7b09d3ded1088aa76924b5812e695317e08c8956dcc07465093a72"} Sep 17 17:48:49.134919 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:49.134882 2586 generic.go:356] "Generic (PLEG): container finished" podID="3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" containerID="a99651648b7b09d3ded1088aa76924b5812e695317e08c8956dcc07465093a72" exitCode=0 Sep 17 17:48:49.135262 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:49.134958 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mvn7n" event={"ID":"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba","Type":"ContainerDied","Data":"a99651648b7b09d3ded1088aa76924b5812e695317e08c8956dcc07465093a72"} Sep 17 17:48:54.160911 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:54.160869 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mvn7n" event={"ID":"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba","Type":"ContainerStarted","Data":"e4a113ef6772905ee48b9f77d1b465a2c894a64b12e429a036975c9511863a2e"} Sep 17 17:48:54.181547 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:48:54.181487 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-mvn7n" podStartSLOduration=1.648201161 podStartE2EDuration="11.181468968s" totalImagesPullingTime="7.750550295s" totalInitContainerRuntime="1.782717512s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:48:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:48:54.180632918 +0000 UTC m=+478.329780328" watchObservedRunningTime="2026-09-17 17:48:54.181468968 +0000 UTC m=+478.330616388" Sep 17 17:49:59.421906 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:49:59.421867 2586 generic.go:356] "Generic (PLEG): container finished" podID="3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" containerID="e4a113ef6772905ee48b9f77d1b465a2c894a64b12e429a036975c9511863a2e" exitCode=0 Sep 17 17:49:59.422315 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:49:59.421930 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mvn7n" event={"ID":"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba","Type":"ContainerDied","Data":"e4a113ef6772905ee48b9f77d1b465a2c894a64b12e429a036975c9511863a2e"} Sep 17 17:50:00.547413 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:00.547391 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-mvn7n" Sep 17 17:50:00.593543 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:00.593517 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjzfm\" (UniqueName: \"kubernetes.io/projected/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-kube-api-access-bjzfm\") pod \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\" (UID: \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\") " Sep 17 17:50:00.593666 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:00.593646 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-model-cache\") pod \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\" (UID: \"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba\") " Sep 17 17:50:00.595812 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:00.595781 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-kube-api-access-bjzfm" (OuterVolumeSpecName: "kube-api-access-bjzfm") pod "3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" (UID: "3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba"). InnerVolumeSpecName "kube-api-access-bjzfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:50:00.612679 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:00.612655 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" (UID: "3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 17:50:00.694851 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:00.694787 2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-model-cache\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:50:00.694851 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:00.694811 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjzfm\" (UniqueName: \"kubernetes.io/projected/3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba-kube-api-access-bjzfm\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:50:01.431706 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:01.431660 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-mvn7n" event={"ID":"3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba","Type":"ContainerDied","Data":"b2215a660108f8485b269f5927653d4e323a28539ea045ee81023441f2580345"} Sep 17 17:50:01.431706 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:01.431713 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2215a660108f8485b269f5927653d4e323a28539ea045ee81023441f2580345" Sep 17 17:50:01.431706 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:01.431677 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-mvn7n" Sep 17 17:50:15.191745 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.191718 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc"] Sep 17 17:50:15.192243 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.191999 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" containerName="s3-init" Sep 17 17:50:15.192243 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.192009 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" containerName="s3-init" Sep 17 17:50:15.192243 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.192026 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" containerName="download-hf-model" Sep 17 17:50:15.192243 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.192032 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" containerName="download-hf-model" Sep 17 17:50:15.192243 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.192078 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba" containerName="s3-init" Sep 17 17:50:15.195178 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.195162 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.199275 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.199254 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Sep 17 17:50:15.199728 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.199711 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gwm88\"" Sep 17 17:50:15.199786 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.199772 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-raw-sklearn-batcher-e3106-predictor-serving-cert\"" Sep 17 17:50:15.200036 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.200021 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 17:50:15.200117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.200056 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 17 17:50:15.200117 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.200074 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-raw-sklearn-batcher-e3106-kube-rbac-proxy-sar-config\"" Sep 17 17:50:15.200493 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.200474 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Sep 17 17:50:15.219875 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.219839 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc"] Sep 17 17:50:15.312079 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.312043 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.312079 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.312073 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.312270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.312100 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-proxy-tls\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.312270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.312160 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.312270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.312202 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rskr7\" (UniqueName: \"kubernetes.io/projected/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-api-access-rskr7\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.413290 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.413253 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.413290 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.413288 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.413491 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.413309 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-proxy-tls\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.413491 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.413341 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.413491 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.413368 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rskr7\" (UniqueName: \"kubernetes.io/projected/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-api-access-rskr7\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.413798 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.413777 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.414078 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.414061 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.414116 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.414085 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.415906 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.415890 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-proxy-tls\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.440805 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.440781 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rskr7\" (UniqueName: \"kubernetes.io/projected/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-api-access-rskr7\") pod \"isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.505334 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.505279 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:15.785234 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:15.785143 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc"] Sep 17 17:50:15.789015 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:50:15.788980 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a4ddb5a_9626_48fa_9f38_ebc2f7b7f6d3.slice/crio-b29144897fca25bfc4b4d549b793c7fb423c30874bffa19c919329eae087f4b4 WatchSource:0}: Error finding container b29144897fca25bfc4b4d549b793c7fb423c30874bffa19c919329eae087f4b4: Status 404 returned error can't find the container with id b29144897fca25bfc4b4d549b793c7fb423c30874bffa19c919329eae087f4b4 Sep 17 17:50:16.492469 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:16.492434 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerStarted","Data":"88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d"} Sep 17 17:50:16.492895 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:16.492493 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerStarted","Data":"b29144897fca25bfc4b4d549b793c7fb423c30874bffa19c919329eae087f4b4"} Sep 17 17:50:20.510291 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:20.510255 2586 generic.go:356] "Generic (PLEG): container finished" podID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerID="88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d" exitCode=0 Sep 17 17:50:20.510728 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:20.510309 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerDied","Data":"88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d"} Sep 17 17:50:34.586157 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:34.586078 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerStarted","Data":"5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff"} Sep 17 17:50:36.597685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:36.597652 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerStarted","Data":"45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48"} Sep 17 17:50:39.614732 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:39.614698 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerStarted","Data":"0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69"} Sep 17 17:50:39.615118 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:39.614968 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:39.615118 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:39.615041 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:39.615118 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:39.615077 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:39.616218 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:39.616190 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:50:39.616834 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:39.616812 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:50:39.635693 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:39.635607 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podStartSLOduration=2.344291846 podStartE2EDuration="24.635594361s" totalImagesPullingTime="18.480284798s" totalInitContainerRuntime="3.811017717s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:50:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:50:39.635476289 +0000 UTC m=+583.784623708" watchObservedRunningTime="2026-09-17 17:50:39.635594361 +0000 UTC m=+583.784741779" Sep 17 17:50:40.618879 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:40.618840 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:50:40.619248 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:40.619228 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:50:45.623820 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:45.623789 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:50:45.624503 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:45.624471 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:50:45.624703 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:45.624681 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:50:55.624972 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:55.624930 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:50:55.625472 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:55.625325 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:50:57.958601 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:57.958573 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:50:57.960882 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:50:57.960860 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:51:05.625194 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:05.625145 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:51:05.625685 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:05.625501 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:51:15.625228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:15.625140 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:51:15.625730 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:15.625653 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:51:25.624495 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:25.624451 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:51:25.624974 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:25.624955 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:51:35.624769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:35.624728 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:51:35.626075 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:35.625233 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:51:45.624791 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:45.624755 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:51:45.625155 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:51:45.624924 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:52:00.039002 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:00.038974 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc"] Sep 17 17:52:00.039441 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:00.039292 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" containerID="cri-o://45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48" gracePeriod=30 Sep 17 17:52:00.039441 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:00.039295 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" containerID="cri-o://5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff" gracePeriod=30 Sep 17 17:52:00.039441 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:00.039288 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" containerID="cri-o://0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69" gracePeriod=30 Sep 17 17:52:00.619794 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:00.619750 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused" Sep 17 17:52:00.933601 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:00.933512 2586 generic.go:356] "Generic (PLEG): container finished" podID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerID="45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48" exitCode=2 Sep 17 17:52:00.933601 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:00.933564 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerDied","Data":"45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48"} Sep 17 17:52:04.952353 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:04.952316 2586 generic.go:356] "Generic (PLEG): container finished" podID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerID="5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff" exitCode=0 Sep 17 17:52:04.952823 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:04.952352 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerDied","Data":"5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff"} Sep 17 17:52:05.619690 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:05.619649 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused" Sep 17 17:52:05.625045 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:05.625022 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:52:05.625460 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:05.625424 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:52:10.619903 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:10.619858 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused" Sep 17 17:52:10.620260 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:10.619994 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:52:15.619000 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:15.618958 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused" Sep 17 17:52:15.624359 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:15.624317 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:52:15.624692 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:15.624667 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:52:20.619422 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:20.619376 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused" Sep 17 17:52:25.619894 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:25.619854 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.25:8643/healthz\": dial tcp 10.132.0.25:8643: connect: connection refused" Sep 17 17:52:25.625280 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:25.625253 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.25:8080: connect: connection refused" Sep 17 17:52:25.625402 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:25.625388 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:52:25.625662 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:25.625638 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:52:25.625745 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:25.625734 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:52:30.180754 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.180729 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:52:30.297863 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.297793 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rskr7\" (UniqueName: \"kubernetes.io/projected/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-api-access-rskr7\") pod \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " Sep 17 17:52:30.297863 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.297836 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-rbac-proxy-sar-config\") pod \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " Sep 17 17:52:30.297863 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.297856 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kserve-provision-location\") pod \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " Sep 17 17:52:30.298116 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.297890 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-proxy-tls\") pod \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " Sep 17 17:52:30.298116 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.297935 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-cabundle-cert\") pod \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\" (UID: \"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3\") " Sep 17 17:52:30.298298 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.298273 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" (UID: "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 17:52:30.298298 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.298280 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" (UID: "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:52:30.298386 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.298315 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" (UID: "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:52:30.300180 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.300159 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-api-access-rskr7" (OuterVolumeSpecName: "kube-api-access-rskr7") pod "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" (UID: "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3"). InnerVolumeSpecName "kube-api-access-rskr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:52:30.300263 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.300174 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" (UID: "5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:52:30.399066 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.399043 2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-cabundle-cert\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:52:30.399066 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.399064 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rskr7\" (UniqueName: \"kubernetes.io/projected/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-api-access-rskr7\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:52:30.399197 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.399075 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:52:30.399197 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.399083 2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-kserve-provision-location\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:52:30.399197 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:30.399093 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3-proxy-tls\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:52:31.060425 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.060386 2586 generic.go:356] "Generic (PLEG): container finished" podID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerID="0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69" exitCode=0 Sep 17 17:52:31.060688 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.060439 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" Sep 17 17:52:31.060688 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.060462 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerDied","Data":"0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69"} Sep 17 17:52:31.060688 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.060524 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc" event={"ID":"5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3","Type":"ContainerDied","Data":"b29144897fca25bfc4b4d549b793c7fb423c30874bffa19c919329eae087f4b4"} Sep 17 17:52:31.060688 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.060562 2586 scope.go:117] "RemoveContainer" containerID="0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69" Sep 17 17:52:31.073003 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.072985 2586 scope.go:117] "RemoveContainer" containerID="45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48" Sep 17 17:52:31.080294 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.080277 2586 scope.go:117] "RemoveContainer" containerID="5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff" Sep 17 17:52:31.086347 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.086323 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc"] Sep 17 17:52:31.087270 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.087258 2586 scope.go:117] "RemoveContainer" containerID="88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d" Sep 17 17:52:31.091198 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.091175 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-raw-sklearn-batcher-e3106-predictor-79ff9bd56f-l8qqc"] Sep 17 17:52:31.095141 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.095126 2586 scope.go:117] "RemoveContainer" containerID="0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69" Sep 17 17:52:31.095408 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:52:31.095390 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69\": container with ID starting with 0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69 not found: ID does not exist" containerID="0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69" Sep 17 17:52:31.095473 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.095420 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69"} err="failed to get container status \"0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69\": rpc error: code = NotFound desc = could not find container \"0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69\": container with ID starting with 0f46384c4c69e7080a4141d0eb62118ffe94a0e01d708d3ce2115cc81da40b69 not found: ID does not exist" Sep 17 17:52:31.095473 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.095444 2586 scope.go:117] "RemoveContainer" containerID="45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48" Sep 17 17:52:31.095747 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:52:31.095730 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48\": container with ID starting with 45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48 not found: ID does not exist" containerID="45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48" Sep 17 17:52:31.095794 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.095752 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48"} err="failed to get container status \"45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48\": rpc error: code = NotFound desc = could not find container \"45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48\": container with ID starting with 45ca921b637d84b69d8cd3daa5ca83b8dd182b666a8e355a9909d476e2e79e48 not found: ID does not exist" Sep 17 17:52:31.095794 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.095775 2586 scope.go:117] "RemoveContainer" containerID="5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff" Sep 17 17:52:31.096016 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:52:31.095999 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff\": container with ID starting with 5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff not found: ID does not exist" containerID="5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff" Sep 17 17:52:31.096056 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.096021 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff"} err="failed to get container status \"5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff\": rpc error: code = NotFound desc = could not find container \"5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff\": container with ID starting with 5a14b8d72897a8fe2d53c1082f565c9d55c7c8eb8261398d5e4d65d8b565c1ff not found: ID does not exist" Sep 17 17:52:31.096056 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.096034 2586 scope.go:117] "RemoveContainer" containerID="88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d" Sep 17 17:52:31.096264 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:52:31.096247 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d\": container with ID starting with 88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d not found: ID does not exist" containerID="88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d" Sep 17 17:52:31.096306 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:31.096268 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d"} err="failed to get container status \"88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d\": rpc error: code = NotFound desc = could not find container \"88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d\": container with ID starting with 88cf45a01580633f43f92beae08ee457bef6cbd792c192e1ecf9a2a8cd4f338d not found: ID does not exist" Sep 17 17:52:32.037836 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:52:32.037799 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" path="/var/lib/kubelet/pods/5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3/volumes" Sep 17 17:53:40.342693 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.342657 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7"] Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.342962 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.342973 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.342992 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.342999 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.343005 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="storage-initializer" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.343011 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="storage-initializer" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.343017 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.343022 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.343064 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kserve-container" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.343073 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="kube-rbac-proxy" Sep 17 17:53:40.343133 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.343080 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a4ddb5a-9626-48fa-9f38-ebc2f7b7f6d3" containerName="agent" Sep 17 17:53:40.346218 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.346201 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.348250 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.348225 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-predictor-serving-cert\"" Sep 17 17:53:40.348370 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.348253 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 17 17:53:40.348593 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.348579 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Sep 17 17:53:40.348997 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.348978 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Sep 17 17:53:40.349258 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.349241 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 17:53:40.350483 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.350466 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gwm88\"" Sep 17 17:53:40.350563 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.350542 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\"" Sep 17 17:53:40.358970 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.358945 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7"] Sep 17 17:53:40.434360 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.434329 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ffea2d3-2969-473c-b13c-18e2dcbd7add-proxy-tls\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.434513 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.434402 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7ggr\" (UniqueName: \"kubernetes.io/projected/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-api-access-k7ggr\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.434513 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.434454 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.434610 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.434530 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kserve-provision-location\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.434610 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.434560 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-cabundle-cert\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.535557 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.535519 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-cabundle-cert\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.535739 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.535576 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ffea2d3-2969-473c-b13c-18e2dcbd7add-proxy-tls\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.535739 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.535601 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7ggr\" (UniqueName: \"kubernetes.io/projected/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-api-access-k7ggr\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.535739 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.535644 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.535739 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.535673 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kserve-provision-location\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.536064 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.536042 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kserve-provision-location\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.536236 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.536218 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-cabundle-cert\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.536301 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.536285 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.538303 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.538286 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ffea2d3-2969-473c-b13c-18e2dcbd7add-proxy-tls\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.543993 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.543974 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7ggr\" (UniqueName: \"kubernetes.io/projected/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-api-access-k7ggr\") pod \"isvc-canary-stop-predictor-646f5bdbcb-jrxw7\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.655719 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.655688 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:40.930422 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:40.930390 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7"] Sep 17 17:53:40.932782 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:53:40.932751 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ffea2d3_2969_473c_b13c_18e2dcbd7add.slice/crio-9b13ebfb1e20f40b0a903cb463bbbc56dec9b342a77d71818acd51956a607961 WatchSource:0}: Error finding container 9b13ebfb1e20f40b0a903cb463bbbc56dec9b342a77d71818acd51956a607961: Status 404 returned error can't find the container with id 9b13ebfb1e20f40b0a903cb463bbbc56dec9b342a77d71818acd51956a607961 Sep 17 17:53:41.325881 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:41.325792 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" event={"ID":"1ffea2d3-2969-473c-b13c-18e2dcbd7add","Type":"ContainerStarted","Data":"e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a"} Sep 17 17:53:41.325881 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:41.325854 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" event={"ID":"1ffea2d3-2969-473c-b13c-18e2dcbd7add","Type":"ContainerStarted","Data":"9b13ebfb1e20f40b0a903cb463bbbc56dec9b342a77d71818acd51956a607961"} Sep 17 17:53:45.344189 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:45.344153 2586 generic.go:356] "Generic (PLEG): container finished" podID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerID="e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a" exitCode=0 Sep 17 17:53:45.344535 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:45.344213 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" event={"ID":"1ffea2d3-2969-473c-b13c-18e2dcbd7add","Type":"ContainerDied","Data":"e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a"} Sep 17 17:53:46.350589 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:46.350548 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" event={"ID":"1ffea2d3-2969-473c-b13c-18e2dcbd7add","Type":"ContainerStarted","Data":"b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f"} Sep 17 17:53:46.351039 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:46.350612 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" event={"ID":"1ffea2d3-2969-473c-b13c-18e2dcbd7add","Type":"ContainerStarted","Data":"fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f"} Sep 17 17:53:46.351039 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:46.350870 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:46.370186 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:46.370137 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podStartSLOduration=2.563547996 podStartE2EDuration="6.370121742s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.806573746s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:53:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:53:46.36935743 +0000 UTC m=+770.518504849" watchObservedRunningTime="2026-09-17 17:53:46.370121742 +0000 UTC m=+770.519269160" Sep 17 17:53:47.354637 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:47.354589 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:47.355874 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:47.355847 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:53:48.357832 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:48.357788 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:53:53.362385 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:53.362356 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:53:53.362908 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:53:53.362883 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:54:03.363309 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:54:03.363268 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:54:13.363279 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:54:13.363193 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:54:23.363769 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:54:23.363725 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:54:33.362877 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:54:33.362835 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:54:43.362873 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:54:43.362833 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:54:53.363291 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:54:53.363246 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Sep 17 17:55:03.364228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:03.364198 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:55:03.935569 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:03.935536 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7"] Sep 17 17:55:03.935804 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:03.935771 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" containerID="cri-o://fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f" gracePeriod=30 Sep 17 17:55:03.935953 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:03.935826 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kube-rbac-proxy" containerID="cri-o://b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f" gracePeriod=30 Sep 17 17:55:04.661806 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:04.661774 2586 generic.go:356] "Generic (PLEG): container finished" podID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerID="b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f" exitCode=2 Sep 17 17:55:04.662187 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:04.661841 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" event={"ID":"1ffea2d3-2969-473c-b13c-18e2dcbd7add","Type":"ContainerDied","Data":"b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f"} Sep 17 17:55:08.358116 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.358079 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.26:8643/healthz\": dial tcp 10.132.0.26:8643: connect: connection refused" Sep 17 17:55:08.469236 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.469208 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:55:08.545281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.545217 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7ggr\" (UniqueName: \"kubernetes.io/projected/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-api-access-k7ggr\") pod \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " Sep 17 17:55:08.545281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.545263 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-cabundle-cert\") pod \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " Sep 17 17:55:08.545420 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.545294 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ffea2d3-2969-473c-b13c-18e2dcbd7add-proxy-tls\") pod \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " Sep 17 17:55:08.545420 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.545326 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-rbac-proxy-sar-config\") pod \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " Sep 17 17:55:08.545420 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.545372 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kserve-provision-location\") pod \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\" (UID: \"1ffea2d3-2969-473c-b13c-18e2dcbd7add\") " Sep 17 17:55:08.545773 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.545743 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1ffea2d3-2969-473c-b13c-18e2dcbd7add" (UID: "1ffea2d3-2969-473c-b13c-18e2dcbd7add"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:55:08.545773 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.545761 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1ffea2d3-2969-473c-b13c-18e2dcbd7add" (UID: "1ffea2d3-2969-473c-b13c-18e2dcbd7add"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 17:55:08.545773 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.545751 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1ffea2d3-2969-473c-b13c-18e2dcbd7add" (UID: "1ffea2d3-2969-473c-b13c-18e2dcbd7add"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:55:08.547432 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.547400 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ffea2d3-2969-473c-b13c-18e2dcbd7add-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1ffea2d3-2969-473c-b13c-18e2dcbd7add" (UID: "1ffea2d3-2969-473c-b13c-18e2dcbd7add"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:55:08.547432 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.547427 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-api-access-k7ggr" (OuterVolumeSpecName: "kube-api-access-k7ggr") pod "1ffea2d3-2969-473c-b13c-18e2dcbd7add" (UID: "1ffea2d3-2969-473c-b13c-18e2dcbd7add"). InnerVolumeSpecName "kube-api-access-k7ggr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:55:08.645893 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.645872 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:55:08.645893 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.645892 2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kserve-provision-location\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:55:08.646020 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.645902 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7ggr\" (UniqueName: \"kubernetes.io/projected/1ffea2d3-2969-473c-b13c-18e2dcbd7add-kube-api-access-k7ggr\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:55:08.646020 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.645912 2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ffea2d3-2969-473c-b13c-18e2dcbd7add-cabundle-cert\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:55:08.646020 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.645920 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ffea2d3-2969-473c-b13c-18e2dcbd7add-proxy-tls\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:55:08.679458 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.679433 2586 generic.go:356] "Generic (PLEG): container finished" podID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerID="fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f" exitCode=0 Sep 17 17:55:08.679553 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.679492 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" Sep 17 17:55:08.679553 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.679508 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" event={"ID":"1ffea2d3-2969-473c-b13c-18e2dcbd7add","Type":"ContainerDied","Data":"fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f"} Sep 17 17:55:08.679643 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.679566 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7" event={"ID":"1ffea2d3-2969-473c-b13c-18e2dcbd7add","Type":"ContainerDied","Data":"9b13ebfb1e20f40b0a903cb463bbbc56dec9b342a77d71818acd51956a607961"} Sep 17 17:55:08.679643 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.679593 2586 scope.go:117] "RemoveContainer" containerID="b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f" Sep 17 17:55:08.690487 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.690463 2586 scope.go:117] "RemoveContainer" containerID="fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f" Sep 17 17:55:08.698051 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.698036 2586 scope.go:117] "RemoveContainer" containerID="e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a" Sep 17 17:55:08.706958 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.706918 2586 scope.go:117] "RemoveContainer" containerID="b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f" Sep 17 17:55:08.707505 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:55:08.707462 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f\": container with ID starting with b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f not found: ID does not exist" containerID="b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f" Sep 17 17:55:08.707598 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.707503 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f"} err="failed to get container status \"b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f\": rpc error: code = NotFound desc = could not find container \"b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f\": container with ID starting with b6f5da69ae5ba23844fff4306a88b7acfcdaec5c820128d4005dbee4f46f9e9f not found: ID does not exist" Sep 17 17:55:08.707598 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.707529 2586 scope.go:117] "RemoveContainer" containerID="fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f" Sep 17 17:55:08.708042 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:55:08.708002 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f\": container with ID starting with fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f not found: ID does not exist" containerID="fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f" Sep 17 17:55:08.708151 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.708037 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f"} err="failed to get container status \"fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f\": rpc error: code = NotFound desc = could not find container \"fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f\": container with ID starting with fed2b9c9a319eb0f77d7de3d01e8f94ea4c9a4bb52e43ebcdd6239a8a6f65e3f not found: ID does not exist" Sep 17 17:55:08.708151 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.708060 2586 scope.go:117] "RemoveContainer" containerID="e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a" Sep 17 17:55:08.708598 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:55:08.708557 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a\": container with ID starting with e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a not found: ID does not exist" containerID="e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a" Sep 17 17:55:08.708971 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.708594 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a"} err="failed to get container status \"e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a\": rpc error: code = NotFound desc = could not find container \"e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a\": container with ID starting with e29c308d1f6f5d1b744c0515387cfeaa876207031508492b836e9969736b473a not found: ID does not exist" Sep 17 17:55:08.710076 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.710050 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7"] Sep 17 17:55:08.712995 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:08.712975 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-stop-predictor-646f5bdbcb-jrxw7"] Sep 17 17:55:10.037381 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:10.037346 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" path="/var/lib/kubelet/pods/1ffea2d3-2969-473c-b13c-18e2dcbd7add/volumes" Sep 17 17:55:57.988087 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:57.988060 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:55:57.988518 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:55:57.988138 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 17:56:00.840537 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840503 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn"] Sep 17 17:56:00.840965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840857 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" Sep 17 17:56:00.840965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840872 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" Sep 17 17:56:00.840965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840889 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="storage-initializer" Sep 17 17:56:00.840965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840894 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="storage-initializer" Sep 17 17:56:00.840965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840900 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kube-rbac-proxy" Sep 17 17:56:00.840965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840906 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kube-rbac-proxy" Sep 17 17:56:00.840965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840958 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kube-rbac-proxy" Sep 17 17:56:00.840965 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.840968 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ffea2d3-2969-473c-b13c-18e2dcbd7add" containerName="kserve-container" Sep 17 17:56:00.843924 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.843909 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:00.845879 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.845850 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Sep 17 17:56:00.845999 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.845890 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Sep 17 17:56:00.846152 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.846134 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 17:56:00.846228 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.846213 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-raw-39c63-predictor-serving-cert\"" Sep 17 17:56:00.846352 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.846338 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 17 17:56:00.846406 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.846393 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-raw-39c63-kube-rbac-proxy-sar-config\"" Sep 17 17:56:00.846628 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.846589 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gwm88\"" Sep 17 17:56:00.857632 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.857598 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn"] Sep 17 17:56:00.939189 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.939151 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kserve-provision-location\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:00.939189 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.939193 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-cabundle-cert\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:00.939404 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.939285 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-proxy-tls\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:00.939404 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.939325 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:00.939404 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:00.939364 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x9hn\" (UniqueName: \"kubernetes.io/projected/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-api-access-2x9hn\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.040257 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.040222 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-cabundle-cert\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.040447 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.040288 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-proxy-tls\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.040447 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.040321 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.040447 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.040367 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x9hn\" (UniqueName: \"kubernetes.io/projected/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-api-access-2x9hn\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.040447 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.040420 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kserve-provision-location\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.040837 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.040814 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kserve-provision-location\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.041043 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.041022 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-cabundle-cert\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.041100 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.041028 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.043004 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.042981 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-proxy-tls\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.048934 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.048913 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x9hn\" (UniqueName: \"kubernetes.io/projected/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-api-access-2x9hn\") pod \"isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.154413 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.154370 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:01.430189 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.430160 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn"] Sep 17 17:56:01.433240 ip-10-0-138-30 kubenswrapper[2586]: W0917 17:56:01.433218 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a297ffb_9e1b_4688_873b_0a9fd2889e8a.slice/crio-3a110bfedf60fa74b37f19fa82ddd37396c01866ecfd0a13fded8e8fa61f5ba4 WatchSource:0}: Error finding container 3a110bfedf60fa74b37f19fa82ddd37396c01866ecfd0a13fded8e8fa61f5ba4: Status 404 returned error can't find the container with id 3a110bfedf60fa74b37f19fa82ddd37396c01866ecfd0a13fded8e8fa61f5ba4 Sep 17 17:56:01.885699 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.885662 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerStarted","Data":"e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf"} Sep 17 17:56:01.886099 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:01.885743 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerStarted","Data":"3a110bfedf60fa74b37f19fa82ddd37396c01866ecfd0a13fded8e8fa61f5ba4"} Sep 17 17:56:05.903441 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:05.903402 2586 generic.go:356] "Generic (PLEG): container finished" podID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerID="e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf" exitCode=0 Sep 17 17:56:05.903813 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:05.903474 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerDied","Data":"e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf"} Sep 17 17:56:06.909510 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.909469 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerStarted","Data":"4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e"} Sep 17 17:56:06.909922 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.909540 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerStarted","Data":"266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4"} Sep 17 17:56:06.909922 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.909566 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerStarted","Data":"dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca"} Sep 17 17:56:06.909922 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.909893 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:06.910070 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.909952 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:06.910070 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.909985 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:06.911457 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.911425 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:56:06.912128 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.912104 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:06.932291 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:06.932248 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podStartSLOduration=3.440431748 podStartE2EDuration="6.932235719s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.491803971s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:06.931047491 +0000 UTC m=+911.080194921" watchObservedRunningTime="2026-09-17 17:56:06.932235719 +0000 UTC m=+911.081383172" Sep 17 17:56:07.914099 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:07.914048 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:56:07.914496 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:07.914471 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:12.918012 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:12.917984 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:56:12.918522 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:12.918492 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:56:12.918921 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:12.918899 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:22.918532 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:22.918479 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:56:22.918927 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:22.918741 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:32.918456 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:32.918417 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:56:32.919065 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:32.918940 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:42.918693 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:42.918648 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:56:42.920920 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:42.919122 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:56:52.918797 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:52.918746 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:56:52.919281 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:56:52.919259 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:02.918802 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:02.918753 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:57:02.919245 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:02.919218 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:12.919262 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:12.919216 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:57:12.919773 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:12.919751 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:20.038002 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:20.037970 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:57:20.038425 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:20.038064 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:57:25.938177 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:25.938139 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn"] Sep 17 17:57:25.938650 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:25.938493 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" containerID="cri-o://dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca" gracePeriod=30 Sep 17 17:57:25.938650 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:25.938498 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" containerID="cri-o://266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4" gracePeriod=30 Sep 17 17:57:25.938788 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:25.938498 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" containerID="cri-o://4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e" gracePeriod=30 Sep 17 17:57:26.225062 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:26.224977 2586 generic.go:356] "Generic (PLEG): container finished" podID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerID="266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4" exitCode=2 Sep 17 17:57:26.225062 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:26.225051 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerDied","Data":"266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4"} Sep 17 17:57:27.914396 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:27.914353 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.27:8643/healthz\": dial tcp 10.132.0.27:8643: connect: connection refused" Sep 17 17:57:30.033416 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:30.033369 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:57:30.033844 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:30.033815 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:31.250487 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:31.250457 2586 generic.go:356] "Generic (PLEG): container finished" podID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerID="dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca" exitCode=0 Sep 17 17:57:31.250858 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:31.250530 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerDied","Data":"dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca"} Sep 17 17:57:32.914720 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:32.914674 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.27:8643/healthz\": dial tcp 10.132.0.27:8643: connect: connection refused" Sep 17 17:57:37.914873 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:37.914827 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.27:8643/healthz\": dial tcp 10.132.0.27:8643: connect: connection refused" Sep 17 17:57:37.915308 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:37.914966 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:57:40.033330 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:40.033277 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:57:40.033760 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:40.033543 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:42.914890 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:42.914837 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.27:8643/healthz\": dial tcp 10.132.0.27:8643: connect: connection refused" Sep 17 17:57:47.915180 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:47.915133 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.27:8643/healthz\": dial tcp 10.132.0.27:8643: connect: connection refused" Sep 17 17:57:50.033357 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:50.033301 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Sep 17 17:57:50.033834 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:50.033689 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 17:57:50.038075 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:50.038055 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:57:50.038182 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:50.038138 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:57:52.914383 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:52.914340 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.27:8643/healthz\": dial tcp 10.132.0.27:8643: connect: connection refused" Sep 17 17:57:56.108145 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.108118 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:57:56.186396 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.186359 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x9hn\" (UniqueName: \"kubernetes.io/projected/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-api-access-2x9hn\") pod \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " Sep 17 17:57:56.186584 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.186417 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-cabundle-cert\") pod \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " Sep 17 17:57:56.186584 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.186450 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-rbac-proxy-sar-config\") pod \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " Sep 17 17:57:56.186584 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.186477 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kserve-provision-location\") pod \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " Sep 17 17:57:56.186584 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.186494 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-proxy-tls\") pod \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\" (UID: \"8a297ffb-9e1b-4688-873b-0a9fd2889e8a\") " Sep 17 17:57:56.186841 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.186818 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8a297ffb-9e1b-4688-873b-0a9fd2889e8a" (UID: "8a297ffb-9e1b-4688-873b-0a9fd2889e8a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 17:57:56.186895 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.186868 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8a297ffb-9e1b-4688-873b-0a9fd2889e8a" (UID: "8a297ffb-9e1b-4688-873b-0a9fd2889e8a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:57:56.186931 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.186871 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "8a297ffb-9e1b-4688-873b-0a9fd2889e8a" (UID: "8a297ffb-9e1b-4688-873b-0a9fd2889e8a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 17:57:56.188823 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.188793 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8a297ffb-9e1b-4688-873b-0a9fd2889e8a" (UID: "8a297ffb-9e1b-4688-873b-0a9fd2889e8a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 17:57:56.188928 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.188842 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-api-access-2x9hn" (OuterVolumeSpecName: "kube-api-access-2x9hn") pod "8a297ffb-9e1b-4688-873b-0a9fd2889e8a" (UID: "8a297ffb-9e1b-4688-873b-0a9fd2889e8a"). InnerVolumeSpecName "kube-api-access-2x9hn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 17:57:56.287919 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.287876 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:57:56.287919 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.287912 2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kserve-provision-location\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:57:56.287919 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.287922 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-proxy-tls\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:57:56.287919 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.287932 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x9hn\" (UniqueName: \"kubernetes.io/projected/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-kube-api-access-2x9hn\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:57:56.288187 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.287940 2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8a297ffb-9e1b-4688-873b-0a9fd2889e8a-cabundle-cert\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 17:57:56.351125 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.351096 2586 generic.go:356] "Generic (PLEG): container finished" podID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerID="4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e" exitCode=137 Sep 17 17:57:56.351258 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.351166 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" Sep 17 17:57:56.351258 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.351178 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerDied","Data":"4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e"} Sep 17 17:57:56.351328 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.351258 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn" event={"ID":"8a297ffb-9e1b-4688-873b-0a9fd2889e8a","Type":"ContainerDied","Data":"3a110bfedf60fa74b37f19fa82ddd37396c01866ecfd0a13fded8e8fa61f5ba4"} Sep 17 17:57:56.351328 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.351304 2586 scope.go:117] "RemoveContainer" containerID="4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e" Sep 17 17:57:56.364998 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.364979 2586 scope.go:117] "RemoveContainer" containerID="266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4" Sep 17 17:57:56.372431 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.372415 2586 scope.go:117] "RemoveContainer" containerID="dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca" Sep 17 17:57:56.377771 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.377749 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn"] Sep 17 17:57:56.379908 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.379883 2586 scope.go:117] "RemoveContainer" containerID="e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf" Sep 17 17:57:56.382572 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.382549 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-39c63-predictor-76f9c7ccf8-hnbkn"] Sep 17 17:57:56.387639 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.387623 2586 scope.go:117] "RemoveContainer" containerID="4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e" Sep 17 17:57:56.387902 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:57:56.387883 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e\": container with ID starting with 4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e not found: ID does not exist" containerID="4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e" Sep 17 17:57:56.387954 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.387913 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e"} err="failed to get container status \"4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e\": rpc error: code = NotFound desc = could not find container \"4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e\": container with ID starting with 4fa9bbd17e76cf73dde8715193a488a7a25ef0b8c021bdab4d863270348ae15e not found: ID does not exist" Sep 17 17:57:56.387954 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.387934 2586 scope.go:117] "RemoveContainer" containerID="266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4" Sep 17 17:57:56.388202 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:57:56.388187 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4\": container with ID starting with 266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4 not found: ID does not exist" containerID="266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4" Sep 17 17:57:56.388287 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.388205 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4"} err="failed to get container status \"266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4\": rpc error: code = NotFound desc = could not find container \"266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4\": container with ID starting with 266f8c5a0a83ae6739b3db623050f54c342e818e89cdcb464652c5bd340d7af4 not found: ID does not exist" Sep 17 17:57:56.388287 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.388221 2586 scope.go:117] "RemoveContainer" containerID="dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca" Sep 17 17:57:56.388466 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:57:56.388449 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca\": container with ID starting with dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca not found: ID does not exist" containerID="dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca" Sep 17 17:57:56.388505 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.388472 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca"} err="failed to get container status \"dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca\": rpc error: code = NotFound desc = could not find container \"dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca\": container with ID starting with dc52e749f29f8521350b2f64476408e77dadc14d4943d99ecf380669ba44adca not found: ID does not exist" Sep 17 17:57:56.388505 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.388487 2586 scope.go:117] "RemoveContainer" containerID="e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf" Sep 17 17:57:56.388749 ip-10-0-138-30 kubenswrapper[2586]: E0917 17:57:56.388735 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf\": container with ID starting with e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf not found: ID does not exist" containerID="e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf" Sep 17 17:57:56.388795 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:56.388752 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf"} err="failed to get container status \"e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf\": rpc error: code = NotFound desc = could not find container \"e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf\": container with ID starting with e7ca56f34640def4a7b0807edfe851ab2badbe48aa98b2e1d1876616234285cf not found: ID does not exist" Sep 17 17:57:58.038399 ip-10-0-138-30 kubenswrapper[2586]: I0917 17:57:58.038370 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" path="/var/lib/kubelet/pods/8a297ffb-9e1b-4688-873b-0a9fd2889e8a/volumes" Sep 17 18:00:58.014719 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:00:58.014689 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 18:00:58.016418 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:00:58.016395 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 18:05:21.120660 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120609 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47"] Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120911 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120921 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120929 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120934 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120949 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="storage-initializer" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120955 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="storage-initializer" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120963 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.120968 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.121014 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kube-rbac-proxy" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.121021 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="agent" Sep 17 18:05:21.121050 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.121028 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a297ffb-9e1b-4688-873b-0a9fd2889e8a" containerName="kserve-container" Sep 17 18:05:21.124195 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.124179 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.128638 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.128594 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-raw-cipn-kube-rbac-proxy-sar-config\"" Sep 17 18:05:21.128759 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.128593 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 18:05:21.128864 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.128845 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-raw-cipn-predictor-serving-cert\"" Sep 17 18:05:21.129029 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.129015 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Sep 17 18:05:21.129871 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.129852 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Sep 17 18:05:21.129871 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.129862 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 17 18:05:21.136894 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.136879 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-gwm88\"" Sep 17 18:05:21.150427 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.150402 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47"] Sep 17 18:05:21.188076 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.188050 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpp2w\" (UniqueName: \"kubernetes.io/projected/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-api-access-tpp2w\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.188182 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.188079 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.188182 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.188125 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.188182 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.188168 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f4188a69-a902-4f3d-9486-766ec34d7ee5-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.188304 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.188212 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4188a69-a902-4f3d-9486-766ec34d7ee5-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.289334 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.289306 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4188a69-a902-4f3d-9486-766ec34d7ee5-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.289451 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.289349 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpp2w\" (UniqueName: \"kubernetes.io/projected/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-api-access-tpp2w\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.289451 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.289372 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.289451 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.289411 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.289451 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.289429 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f4188a69-a902-4f3d-9486-766ec34d7ee5-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.289827 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.289807 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f4188a69-a902-4f3d-9486-766ec34d7ee5-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.290143 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.290124 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.290248 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.290229 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.291922 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.291902 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4188a69-a902-4f3d-9486-766ec34d7ee5-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.299276 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.299254 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpp2w\" (UniqueName: \"kubernetes.io/projected/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-api-access-tpp2w\") pod \"isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.436913 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.436809 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:21.716594 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:21.716561 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47"] Sep 17 18:05:21.718108 ip-10-0-138-30 kubenswrapper[2586]: W0917 18:05:21.718084 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4188a69_a902_4f3d_9486_766ec34d7ee5.slice/crio-fa951053e35b0299da190398929531936f48dc77ef0003c6a2192214db0e97cc WatchSource:0}: Error finding container fa951053e35b0299da190398929531936f48dc77ef0003c6a2192214db0e97cc: Status 404 returned error can't find the container with id fa951053e35b0299da190398929531936f48dc77ef0003c6a2192214db0e97cc Sep 17 18:05:22.112509 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:22.112475 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerStarted","Data":"6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0"} Sep 17 18:05:22.112706 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:22.112538 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerStarted","Data":"fa951053e35b0299da190398929531936f48dc77ef0003c6a2192214db0e97cc"} Sep 17 18:05:26.129852 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:26.129821 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerID="6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0" exitCode=0 Sep 17 18:05:26.130209 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:26.129894 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerDied","Data":"6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0"} Sep 17 18:05:27.136723 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:27.136674 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerStarted","Data":"ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9"} Sep 17 18:05:27.137191 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:27.136769 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerStarted","Data":"4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4"} Sep 17 18:05:27.137191 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:27.136808 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerStarted","Data":"40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451"} Sep 17 18:05:27.137191 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:27.137066 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:27.172334 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:27.172278 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podStartSLOduration=2.331253137 podStartE2EDuration="6.172265047s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.84101191s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:05:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:05:27.171565132 +0000 UTC m=+1471.320712591" watchObservedRunningTime="2026-09-17 18:05:27.172265047 +0000 UTC m=+1471.321412467" Sep 17 18:05:28.140890 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:28.140850 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:28.141290 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:28.140932 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:28.142332 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:28.142307 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:05:28.143029 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:28.142993 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:29.145078 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:29.145029 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:05:29.145534 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:29.145508 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:34.149537 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:34.149511 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:05:34.150242 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:34.150205 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:05:34.150550 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:34.150528 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:44.150360 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:44.150312 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:05:44.150855 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:44.150833 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:54.150782 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:54.150738 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:05:54.151284 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:54.151133 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:05:58.048802 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:58.048771 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 18:05:58.051652 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:05:58.051610 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 18:06:04.150367 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:04.150323 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:06:04.150795 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:04.150760 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:14.150871 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:14.150771 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:06:14.151308 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:14.151284 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:24.150174 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:24.150126 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:06:24.150591 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:24.150507 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:34.150117 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:34.150069 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:06:34.150634 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:34.150503 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:06:44.150676 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:44.150609 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:06:44.151065 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:44.150777 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:06:56.440101 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:56.440064 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47"] Sep 17 18:06:56.440465 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:56.440372 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" containerID="cri-o://40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451" gracePeriod=30 Sep 17 18:06:56.440465 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:56.440434 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" containerID="cri-o://4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4" gracePeriod=30 Sep 17 18:06:56.440602 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:56.440426 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" containerID="cri-o://ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9" gracePeriod=30 Sep 17 18:06:57.497944 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:57.497903 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerID="4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4" exitCode=2 Sep 17 18:06:57.498326 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:57.497975 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerDied","Data":"4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4"} Sep 17 18:06:59.145993 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:06:59.145954 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused" Sep 17 18:07:01.515430 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:01.515396 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerID="40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451" exitCode=0 Sep 17 18:07:01.515801 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:01.515466 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerDied","Data":"40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451"} Sep 17 18:07:04.146017 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:04.145970 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused" Sep 17 18:07:04.150369 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:04.150333 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:07:04.150692 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:04.150661 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:09.145340 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:09.145300 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused" Sep 17 18:07:09.145806 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:09.145425 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:07:13.001334 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:13.001306 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:13.003796 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:13.003779 2586 ???:1] "http: TLS handshake error from 10.0.136.127:33398: EOF" Sep 17 18:07:13.047325 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:13.047298 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:13.063156 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:13.063130 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:13.078472 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:13.078452 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:13.981452 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:13.981424 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:13.997304 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:13.997286 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:14.011189 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.011170 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:14.026887 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.026870 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:14.146020 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.145988 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused" Sep 17 18:07:14.150333 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.150309 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:07:14.150681 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.150658 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:14.940946 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.940912 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:14.958787 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.958762 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:14.972857 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.972829 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:14.986385 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:14.986367 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:15.854292 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:15.854265 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:15.871776 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:15.871757 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:15.884590 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:15.884571 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:15.899178 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:15.899136 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:16.775079 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:16.775052 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:16.791673 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:16.791640 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:16.812900 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:16.812872 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:16.830923 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:16.830904 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:17.730734 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:17.730702 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:17.745427 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:17.745401 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:17.757185 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:17.757164 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:17.771099 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:17.771080 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:18.641782 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:18.641752 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:18.657110 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:18.657088 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:18.670144 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:18.670126 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:18.683725 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:18.683708 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:19.145758 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:19.145717 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused" Sep 17 18:07:19.556363 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:19.556294 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:19.572671 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:19.572648 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:19.586333 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:19.586311 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:19.603384 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:19.603366 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:20.478167 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:20.478143 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:20.492654 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:20.492634 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:20.505785 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:20.505764 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:20.518822 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:20.518804 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:21.377320 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:21.377295 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:21.393487 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:21.393462 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:21.409107 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:21.409088 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:21.423101 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:21.423083 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:22.346288 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:22.346257 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:22.362272 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:22.362251 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:22.383293 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:22.383273 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:22.397514 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:22.397492 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:23.307377 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:23.307340 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:23.321316 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:23.321294 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:23.335131 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:23.335110 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:23.348494 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:23.348476 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:24.145808 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.145773 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused" Sep 17 18:07:24.150089 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.150050 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused" Sep 17 18:07:24.150236 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.150193 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:07:24.150405 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.150379 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 18:07:24.150497 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.150483 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:07:24.263933 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.263899 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:24.279001 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.278976 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:24.291657 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.291640 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:24.304439 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:24.304412 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:25.206010 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:25.205982 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:25.220274 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:25.220255 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:25.237064 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:25.237045 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:25.249029 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:25.249012 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:26.183145 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.183115 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kserve-container/0.log" Sep 17 18:07:26.197673 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.197650 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/kube-rbac-proxy/0.log" Sep 17 18:07:26.218123 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.218104 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/agent/0.log" Sep 17 18:07:26.231478 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.231454 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47_f4188a69-a902-4f3d-9486-766ec34d7ee5/storage-initializer/0.log" Sep 17 18:07:26.582586 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.582561 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:07:26.616939 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.616909 2586 generic.go:356] "Generic (PLEG): container finished" podID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerID="ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9" exitCode=0 Sep 17 18:07:26.617040 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.616971 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" Sep 17 18:07:26.617040 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.616979 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerDied","Data":"ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9"} Sep 17 18:07:26.617120 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.617049 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47" event={"ID":"f4188a69-a902-4f3d-9486-766ec34d7ee5","Type":"ContainerDied","Data":"fa951053e35b0299da190398929531936f48dc77ef0003c6a2192214db0e97cc"} Sep 17 18:07:26.617120 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.617108 2586 scope.go:117] "RemoveContainer" containerID="ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9" Sep 17 18:07:26.625336 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.625320 2586 scope.go:117] "RemoveContainer" containerID="4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4" Sep 17 18:07:26.632809 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.632788 2586 scope.go:117] "RemoveContainer" containerID="40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451" Sep 17 18:07:26.639505 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.639489 2586 scope.go:117] "RemoveContainer" containerID="6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0" Sep 17 18:07:26.646313 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.646286 2586 scope.go:117] "RemoveContainer" containerID="ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9" Sep 17 18:07:26.646560 ip-10-0-138-30 kubenswrapper[2586]: E0917 18:07:26.646542 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9\": container with ID starting with ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9 not found: ID does not exist" containerID="ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9" Sep 17 18:07:26.646637 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.646567 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9"} err="failed to get container status \"ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9\": rpc error: code = NotFound desc = could not find container \"ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9\": container with ID starting with ee3333d71d5df30decf00399b581aebbda39b00d990b9d8313e4be221cd6a9a9 not found: ID does not exist" Sep 17 18:07:26.646637 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.646582 2586 scope.go:117] "RemoveContainer" containerID="4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4" Sep 17 18:07:26.646863 ip-10-0-138-30 kubenswrapper[2586]: E0917 18:07:26.646844 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4\": container with ID starting with 4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4 not found: ID does not exist" containerID="4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4" Sep 17 18:07:26.646914 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.646872 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4"} err="failed to get container status \"4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4\": rpc error: code = NotFound desc = could not find container \"4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4\": container with ID starting with 4efa8b558f9c7304fdfc7c85eadd48ca2558f3e27ed05ec9957f4064c48fb9c4 not found: ID does not exist" Sep 17 18:07:26.646914 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.646890 2586 scope.go:117] "RemoveContainer" containerID="40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451" Sep 17 18:07:26.647135 ip-10-0-138-30 kubenswrapper[2586]: E0917 18:07:26.647118 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451\": container with ID starting with 40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451 not found: ID does not exist" containerID="40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451" Sep 17 18:07:26.647198 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.647146 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451"} err="failed to get container status \"40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451\": rpc error: code = NotFound desc = could not find container \"40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451\": container with ID starting with 40a69df89fc6b9c47e242fdea1aff2d547bb7aeeddb5ffcf4292be4a4bd36451 not found: ID does not exist" Sep 17 18:07:26.647198 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.647168 2586 scope.go:117] "RemoveContainer" containerID="6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0" Sep 17 18:07:26.647458 ip-10-0-138-30 kubenswrapper[2586]: E0917 18:07:26.647442 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0\": container with ID starting with 6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0 not found: ID does not exist" containerID="6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0" Sep 17 18:07:26.647499 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.647463 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0"} err="failed to get container status \"6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0\": rpc error: code = NotFound desc = could not find container \"6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0\": container with ID starting with 6c16e6ecbd52770e8720ad2c679e6d64bec9d24660a71d7cfe5fb29bf00231b0 not found: ID does not exist" Sep 17 18:07:26.663355 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.663339 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4188a69-a902-4f3d-9486-766ec34d7ee5-proxy-tls\") pod \"f4188a69-a902-4f3d-9486-766ec34d7ee5\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " Sep 17 18:07:26.663444 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.663402 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpp2w\" (UniqueName: \"kubernetes.io/projected/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-api-access-tpp2w\") pod \"f4188a69-a902-4f3d-9486-766ec34d7ee5\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " Sep 17 18:07:26.663444 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.663420 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-rbac-proxy-sar-config\") pod \"f4188a69-a902-4f3d-9486-766ec34d7ee5\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " Sep 17 18:07:26.663444 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.663434 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-cabundle-cert\") pod \"f4188a69-a902-4f3d-9486-766ec34d7ee5\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " Sep 17 18:07:26.663598 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.663464 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f4188a69-a902-4f3d-9486-766ec34d7ee5-kserve-provision-location\") pod \"f4188a69-a902-4f3d-9486-766ec34d7ee5\" (UID: \"f4188a69-a902-4f3d-9486-766ec34d7ee5\") " Sep 17 18:07:26.663836 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.663814 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "f4188a69-a902-4f3d-9486-766ec34d7ee5" (UID: "f4188a69-a902-4f3d-9486-766ec34d7ee5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 18:07:26.663836 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.663813 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f4188a69-a902-4f3d-9486-766ec34d7ee5" (UID: "f4188a69-a902-4f3d-9486-766ec34d7ee5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 18:07:26.663950 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.663848 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4188a69-a902-4f3d-9486-766ec34d7ee5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "f4188a69-a902-4f3d-9486-766ec34d7ee5" (UID: "f4188a69-a902-4f3d-9486-766ec34d7ee5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 18:07:26.665443 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.665425 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4188a69-a902-4f3d-9486-766ec34d7ee5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f4188a69-a902-4f3d-9486-766ec34d7ee5" (UID: "f4188a69-a902-4f3d-9486-766ec34d7ee5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 18:07:26.665549 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.665531 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-api-access-tpp2w" (OuterVolumeSpecName: "kube-api-access-tpp2w") pod "f4188a69-a902-4f3d-9486-766ec34d7ee5" (UID: "f4188a69-a902-4f3d-9486-766ec34d7ee5"). InnerVolumeSpecName "kube-api-access-tpp2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 18:07:26.764648 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.764572 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4188a69-a902-4f3d-9486-766ec34d7ee5-proxy-tls\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 18:07:26.764648 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.764595 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpp2w\" (UniqueName: \"kubernetes.io/projected/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-api-access-tpp2w\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 18:07:26.764648 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.764605 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 18:07:26.764648 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.764633 2586 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f4188a69-a902-4f3d-9486-766ec34d7ee5-cabundle-cert\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 18:07:26.764648 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.764642 2586 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f4188a69-a902-4f3d-9486-766ec34d7ee5-kserve-provision-location\") on node \"ip-10-0-138-30.ec2.internal\" DevicePath \"\"" Sep 17 18:07:26.944424 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.944393 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47"] Sep 17 18:07:26.947734 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:26.947713 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-raw-cipn-predictor-7c9c6744b4-tcl47"] Sep 17 18:07:28.037882 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.037849 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" path="/var/lib/kubelet/pods/f4188a69-a902-4f3d-9486-766ec34d7ee5/volumes" Sep 17 18:07:28.544263 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544234 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-66kd5/must-gather-59jnb"] Sep 17 18:07:28.544543 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544531 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" Sep 17 18:07:28.544587 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544545 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" Sep 17 18:07:28.544587 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544558 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="storage-initializer" Sep 17 18:07:28.544587 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544563 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="storage-initializer" Sep 17 18:07:28.544587 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544575 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" Sep 17 18:07:28.544587 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544580 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" Sep 17 18:07:28.544587 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544585 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" Sep 17 18:07:28.544786 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544592 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" Sep 17 18:07:28.544786 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544653 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kube-rbac-proxy" Sep 17 18:07:28.544786 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544662 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="agent" Sep 17 18:07:28.544786 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.544667 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4188a69-a902-4f3d-9486-766ec34d7ee5" containerName="kserve-container" Sep 17 18:07:28.549308 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.549289 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-66kd5/must-gather-59jnb" Sep 17 18:07:28.551844 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.551821 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-66kd5\"/\"default-dockercfg-frhg2\"" Sep 17 18:07:28.551962 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.551944 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-66kd5\"/\"kube-root-ca.crt\"" Sep 17 18:07:28.552032 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.551971 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-66kd5\"/\"openshift-service-ca.crt\"" Sep 17 18:07:28.557240 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.557222 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-66kd5/must-gather-59jnb"] Sep 17 18:07:28.578708 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.578685 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dca5135c-1d03-45a3-9a91-54aa718f80dd-must-gather-output\") pod \"must-gather-59jnb\" (UID: \"dca5135c-1d03-45a3-9a91-54aa718f80dd\") " pod="openshift-must-gather-66kd5/must-gather-59jnb" Sep 17 18:07:28.578815 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.578761 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnwjh\" (UniqueName: \"kubernetes.io/projected/dca5135c-1d03-45a3-9a91-54aa718f80dd-kube-api-access-rnwjh\") pod \"must-gather-59jnb\" (UID: \"dca5135c-1d03-45a3-9a91-54aa718f80dd\") " pod="openshift-must-gather-66kd5/must-gather-59jnb" Sep 17 18:07:28.679490 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.679462 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dca5135c-1d03-45a3-9a91-54aa718f80dd-must-gather-output\") pod \"must-gather-59jnb\" (UID: \"dca5135c-1d03-45a3-9a91-54aa718f80dd\") " pod="openshift-must-gather-66kd5/must-gather-59jnb" Sep 17 18:07:28.679603 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.679536 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnwjh\" (UniqueName: \"kubernetes.io/projected/dca5135c-1d03-45a3-9a91-54aa718f80dd-kube-api-access-rnwjh\") pod \"must-gather-59jnb\" (UID: \"dca5135c-1d03-45a3-9a91-54aa718f80dd\") " pod="openshift-must-gather-66kd5/must-gather-59jnb" Sep 17 18:07:28.679886 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.679870 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/dca5135c-1d03-45a3-9a91-54aa718f80dd-must-gather-output\") pod \"must-gather-59jnb\" (UID: \"dca5135c-1d03-45a3-9a91-54aa718f80dd\") " pod="openshift-must-gather-66kd5/must-gather-59jnb" Sep 17 18:07:28.689480 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.689459 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnwjh\" (UniqueName: \"kubernetes.io/projected/dca5135c-1d03-45a3-9a91-54aa718f80dd-kube-api-access-rnwjh\") pod \"must-gather-59jnb\" (UID: \"dca5135c-1d03-45a3-9a91-54aa718f80dd\") " pod="openshift-must-gather-66kd5/must-gather-59jnb" Sep 17 18:07:28.867133 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:28.867107 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-66kd5/must-gather-59jnb" Sep 17 18:07:29.155704 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:29.155632 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-66kd5/must-gather-59jnb"] Sep 17 18:07:29.158704 ip-10-0-138-30 kubenswrapper[2586]: W0917 18:07:29.158679 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddca5135c_1d03_45a3_9a91_54aa718f80dd.slice/crio-0c0beb084ec8dc8a14bdde2cf3d0de7f426c06f4d201b2538214db1cef5f3048 WatchSource:0}: Error finding container 0c0beb084ec8dc8a14bdde2cf3d0de7f426c06f4d201b2538214db1cef5f3048: Status 404 returned error can't find the container with id 0c0beb084ec8dc8a14bdde2cf3d0de7f426c06f4d201b2538214db1cef5f3048 Sep 17 18:07:29.160570 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:29.160548 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 18:07:29.630926 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:29.630895 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-66kd5/must-gather-59jnb" event={"ID":"dca5135c-1d03-45a3-9a91-54aa718f80dd","Type":"ContainerStarted","Data":"0c0beb084ec8dc8a14bdde2cf3d0de7f426c06f4d201b2538214db1cef5f3048"} Sep 17 18:07:30.637285 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:30.637240 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-66kd5/must-gather-59jnb" event={"ID":"dca5135c-1d03-45a3-9a91-54aa718f80dd","Type":"ContainerStarted","Data":"5102a4176145707326e2022e59039c4e7daff9ede04b01b94cfa4bd4c44c1f83"} Sep 17 18:07:30.637722 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:30.637300 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-66kd5/must-gather-59jnb" event={"ID":"dca5135c-1d03-45a3-9a91-54aa718f80dd","Type":"ContainerStarted","Data":"7a87a21f8931ed85cabb6b45a6ddc41c890fbf161467e56db8f4b70d7a60f4c4"} Sep 17 18:07:30.666670 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:30.666604 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-66kd5/must-gather-59jnb" podStartSLOduration=1.857188338 podStartE2EDuration="2.666586466s" totalImagesPullingTime="809.398128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:07:30.665162114 +0000 UTC m=+1594.814309534" watchObservedRunningTime="2026-09-17 18:07:30.666586466 +0000 UTC m=+1594.815733887" Sep 17 18:07:31.646508 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:31.646477 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xs9vn_d3e728a9-fb95-4223-9920-37a2972a101c/global-pull-secret-syncer/0.log" Sep 17 18:07:31.767499 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:31.767466 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mz7nt_3520ab4e-6be6-42fb-888f-cbd57c6e7506/konnectivity-agent/0.log" Sep 17 18:07:31.851113 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:31.851069 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-30.ec2.internal_06841d19bfd80202ae04f77c7199dad4/haproxy/0.log" Sep 17 18:07:36.373160 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:36.373126 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-f4hg9_97f023e7-bed4-4556-8279-20e645880b88/cluster-monitoring-operator/0.log" Sep 17 18:07:36.694324 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:36.694295 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5pfsf_6a11b0c2-a065-458f-876c-15861ee77e8b/node-exporter/0.log" Sep 17 18:07:36.723357 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:36.723329 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5pfsf_6a11b0c2-a065-458f-876c-15861ee77e8b/kube-rbac-proxy/0.log" Sep 17 18:07:36.752162 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:36.752135 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5pfsf_6a11b0c2-a065-458f-876c-15861ee77e8b/init-textfile/0.log" Sep 17 18:07:36.989963 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:36.989884 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53a68631-6c30-46f8-b5c2-009d00ab0cda/prometheus/0.log" Sep 17 18:07:37.019438 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:37.019401 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53a68631-6c30-46f8-b5c2-009d00ab0cda/config-reloader/0.log" Sep 17 18:07:37.047245 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:37.047138 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53a68631-6c30-46f8-b5c2-009d00ab0cda/thanos-sidecar/0.log" Sep 17 18:07:37.075430 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:37.075318 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53a68631-6c30-46f8-b5c2-009d00ab0cda/kube-rbac-proxy-web/0.log" Sep 17 18:07:37.110375 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:37.110338 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53a68631-6c30-46f8-b5c2-009d00ab0cda/kube-rbac-proxy/0.log" Sep 17 18:07:37.142131 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:37.142077 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53a68631-6c30-46f8-b5c2-009d00ab0cda/kube-rbac-proxy-thanos/0.log" Sep 17 18:07:37.186229 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:37.186195 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_53a68631-6c30-46f8-b5c2-009d00ab0cda/init-config-reloader/0.log" Sep 17 18:07:37.223450 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:37.223400 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-zxjq5_46659d9e-fe98-4c53-8d6d-227825b7c492/prometheus-operator/0.log" Sep 17 18:07:37.252451 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:37.252368 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-zxjq5_46659d9e-fe98-4c53-8d6d-227825b7c492/kube-rbac-proxy/0.log" Sep 17 18:07:38.809028 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:38.809001 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-sp2x7_e37515af-75c0-4ee7-8552-5212fc707e1f/networking-console-plugin/0.log" Sep 17 18:07:39.293643 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.293587 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/1.log" Sep 17 18:07:39.303039 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.303012 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-wb4t7_9ed0c96e-8722-4474-9499-d56cb6a3df9c/console-operator/2.log" Sep 17 18:07:39.536278 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.536235 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8"] Sep 17 18:07:39.539760 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.539736 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.569074 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.569039 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8"] Sep 17 18:07:39.686886 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.686847 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qqhx\" (UniqueName: \"kubernetes.io/projected/921168e1-9408-42e7-bf47-3b31ca7c45a0-kube-api-access-7qqhx\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.687070 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.686968 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-sys\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.687070 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.687000 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-lib-modules\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.687070 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.687041 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-proc\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.687070 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.687064 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-podres\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788280 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788232 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-proc\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788481 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788295 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-podres\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788481 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788328 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qqhx\" (UniqueName: \"kubernetes.io/projected/921168e1-9408-42e7-bf47-3b31ca7c45a0-kube-api-access-7qqhx\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788481 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788432 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-sys\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788481 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788463 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-lib-modules\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788742 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788688 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-lib-modules\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788800 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788765 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-proc\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788867 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788850 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-podres\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.788975 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.788957 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/921168e1-9408-42e7-bf47-3b31ca7c45a0-sys\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.799429 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.799340 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qqhx\" (UniqueName: \"kubernetes.io/projected/921168e1-9408-42e7-bf47-3b31ca7c45a0-kube-api-access-7qqhx\") pod \"perf-node-gather-daemonset-fjrv8\" (UID: \"921168e1-9408-42e7-bf47-3b31ca7c45a0\") " pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.850649 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.850585 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:39.927745 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:39.925876 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-856ks_8312e663-4022-4dde-9aab-2d9e52c2c8d8/download-server/0.log" Sep 17 18:07:40.138954 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:40.138924 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8"] Sep 17 18:07:40.141574 ip-10-0-138-30 kubenswrapper[2586]: W0917 18:07:40.141541 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod921168e1_9408_42e7_bf47_3b31ca7c45a0.slice/crio-b019fda6290aebcd7b64f4b4e913f0778f2cbe0b4825d3b4db282b83a2d0dbdd WatchSource:0}: Error finding container b019fda6290aebcd7b64f4b4e913f0778f2cbe0b4825d3b4db282b83a2d0dbdd: Status 404 returned error can't find the container with id b019fda6290aebcd7b64f4b4e913f0778f2cbe0b4825d3b4db282b83a2d0dbdd Sep 17 18:07:40.507395 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:40.507327 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-zkz22_9d1b0622-6a48-4266-8066-8db5560a77b2/volume-data-source-validator/0.log" Sep 17 18:07:40.693604 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:40.693556 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" event={"ID":"921168e1-9408-42e7-bf47-3b31ca7c45a0","Type":"ContainerStarted","Data":"ba18c082fdf6934bb31ce451b0ab400ed251957935cef2c8dbfefd62f17dc528"} Sep 17 18:07:40.693792 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:40.693651 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" event={"ID":"921168e1-9408-42e7-bf47-3b31ca7c45a0","Type":"ContainerStarted","Data":"b019fda6290aebcd7b64f4b4e913f0778f2cbe0b4825d3b4db282b83a2d0dbdd"} Sep 17 18:07:40.694216 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:40.694189 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:40.715385 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:40.715337 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" podStartSLOduration=1.715319515 podStartE2EDuration="1.715319515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 18:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:07:40.714390578 +0000 UTC m=+1604.863538000" watchObservedRunningTime="2026-09-17 18:07:40.715319515 +0000 UTC m=+1604.864466934" Sep 17 18:07:41.526059 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:41.526028 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sshtr_ed64f380-f001-47b9-917e-990dd6ea473f/dns/0.log" Sep 17 18:07:41.553454 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:41.553430 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sshtr_ed64f380-f001-47b9-917e-990dd6ea473f/kube-rbac-proxy/0.log" Sep 17 18:07:41.631347 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:41.631318 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kz957_4d0ec105-91bf-460a-9fcf-307246b66218/dns-node-resolver/0.log" Sep 17 18:07:42.215726 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:42.215700 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tnn9m_6d3c9ef5-2029-476f-bc07-5cc54bc5ad18/node-ca/0.log" Sep 17 18:07:43.031272 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:43.031247 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6cb8c868-qgggf_9298287b-3797-48a7-995b-f613902a2a98/router/0.log" Sep 17 18:07:43.445643 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:43.445603 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dpn22_d09443ff-a2b4-4a53-861f-810d4d2cd4f3/serve-healthcheck-canary/0.log" Sep 17 18:07:43.886335 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:43.886304 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-lzhcp_19467e99-0315-4947-9934-80d423a3e33d/insights-operator/0.log" Sep 17 18:07:43.886609 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:43.886593 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-lzhcp_19467e99-0315-4947-9934-80d423a3e33d/insights-operator/1.log" Sep 17 18:07:44.001735 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:44.001706 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8l44_3983db55-61ff-4e21-bfbb-b152cdd0eb9a/kube-rbac-proxy/0.log" Sep 17 18:07:44.029026 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:44.028998 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8l44_3983db55-61ff-4e21-bfbb-b152cdd0eb9a/exporter/0.log" Sep 17 18:07:44.057164 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:44.057136 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8l44_3983db55-61ff-4e21-bfbb-b152cdd0eb9a/extractor/0.log" Sep 17 18:07:46.686364 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:46.686337 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-mvn7n_3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba/s3-init/0.log" Sep 17 18:07:46.708912 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:46.708886 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-mvn7n_3ec2c2ce-f60d-4dc4-8fe5-286fd989b5ba/download-hf-model/0.log" Sep 17 18:07:47.717884 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:47.717850 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-66kd5/perf-node-gather-daemonset-fjrv8" Sep 17 18:07:51.903431 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:51.903360 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-nhxzc_778a9f61-f31c-4157-9e8c-69c40f0dd7a3/kube-storage-version-migrator-operator/1.log" Sep 17 18:07:51.904738 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:51.904713 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-nhxzc_778a9f61-f31c-4157-9e8c-69c40f0dd7a3/kube-storage-version-migrator-operator/0.log" Sep 17 18:07:53.324579 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.324548 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pcvrb_9eedb140-05c7-48ce-8171-13998ef8be9e/kube-multus-additional-cni-plugins/0.log" Sep 17 18:07:53.350899 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.350854 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pcvrb_9eedb140-05c7-48ce-8171-13998ef8be9e/egress-router-binary-copy/0.log" Sep 17 18:07:53.376721 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.376691 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pcvrb_9eedb140-05c7-48ce-8171-13998ef8be9e/cni-plugins/0.log" Sep 17 18:07:53.406046 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.406020 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pcvrb_9eedb140-05c7-48ce-8171-13998ef8be9e/bond-cni-plugin/0.log" Sep 17 18:07:53.430334 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.430307 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pcvrb_9eedb140-05c7-48ce-8171-13998ef8be9e/routeoverride-cni/0.log" Sep 17 18:07:53.455729 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.455708 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pcvrb_9eedb140-05c7-48ce-8171-13998ef8be9e/whereabouts-cni-bincopy/0.log" Sep 17 18:07:53.482472 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.482448 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pcvrb_9eedb140-05c7-48ce-8171-13998ef8be9e/whereabouts-cni/0.log" Sep 17 18:07:53.550394 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.550370 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-v56s5_86df2dae-c4b8-4042-a75a-29e68a739826/kube-multus/0.log" Sep 17 18:07:53.720084 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.720056 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xnrfz_dd1e8497-38f8-4cf2-b979-8cbadd3f4d38/network-metrics-daemon/0.log" Sep 17 18:07:53.751254 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:53.751201 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xnrfz_dd1e8497-38f8-4cf2-b979-8cbadd3f4d38/kube-rbac-proxy/0.log" Sep 17 18:07:54.589570 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:54.589543 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6mvdd_720b09de-1c22-4a55-88f6-66aca41add21/ovn-controller/0.log" Sep 17 18:07:54.611510 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:54.611490 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6mvdd_720b09de-1c22-4a55-88f6-66aca41add21/ovn-acl-logging/0.log" Sep 17 18:07:54.644339 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:54.644313 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6mvdd_720b09de-1c22-4a55-88f6-66aca41add21/kube-rbac-proxy-node/0.log" Sep 17 18:07:54.671952 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:54.671920 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6mvdd_720b09de-1c22-4a55-88f6-66aca41add21/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 18:07:54.697960 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:54.697938 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6mvdd_720b09de-1c22-4a55-88f6-66aca41add21/northd/0.log" Sep 17 18:07:54.724147 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:54.724118 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6mvdd_720b09de-1c22-4a55-88f6-66aca41add21/nbdb/0.log" Sep 17 18:07:54.752859 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:54.752837 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6mvdd_720b09de-1c22-4a55-88f6-66aca41add21/sbdb/0.log" Sep 17 18:07:54.796839 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:54.796817 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6mvdd_720b09de-1c22-4a55-88f6-66aca41add21/ovnkube-controller/0.log" Sep 17 18:07:56.362491 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:56.362458 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7cbc45fddc-wt2kt_c85a02d3-2fba-4bde-b739-2486d2a637a4/check-endpoints/0.log" Sep 17 18:07:56.451831 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:56.451804 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-wf9fj_de127252-05c1-43c0-aba1-095c828851f9/network-check-target-container/0.log" Sep 17 18:07:57.559332 ip-10-0-138-30 kubenswrapper[2586]: I0917 18:07:57.559265 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8spf2_7191de83-8ab8-4a34-91ec-2bc5fb95368b/iptables-alerter/0.log"