Jun 02 15:43:54.470892 ip-10-0-128-123 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 02 15:43:54.470904 ip-10-0-128-123 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 02 15:43:54.470914 ip-10-0-128-123 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 02 15:43:54.471243 ip-10-0-128-123 systemd[1]: Failed to start Kubernetes Kubelet. Jun 02 15:44:04.543532 ip-10-0-128-123 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 02 15:44:04.543550 ip-10-0-128-123 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d8ab51d877354dfb841f7f0fba4aba3b -- Jun 02 15:46:28.756703 ip-10-0-128-123 systemd[1]: Starting Kubernetes Kubelet... Jun 02 15:46:29.194044 ip-10-0-128-123 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 15:46:29.194044 ip-10-0-128-123 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 15:46:29.194044 ip-10-0-128-123 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 15:46:29.194044 ip-10-0-128-123 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 15:46:29.194044 ip-10-0-128-123 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 15:46:29.194718 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.194610 2578 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 15:46:29.200082 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200061 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:46:29.200082 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200080 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:46:29.200082 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200084 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200090 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200093 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200096 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200099 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200102 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200105 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200108 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200111 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200113 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200116 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200119 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200121 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200124 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200127 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200129 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200132 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200135 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200137 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200140 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:46:29.200184 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200142 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200147 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200151 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200154 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200158 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200160 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200171 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200174 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200177 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200179 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200182 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200184 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200187 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200191 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200194 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200197 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200200 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200203 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200205 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:46:29.200671 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200209 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200211 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200214 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200216 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200219 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200223 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200227 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200230 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200232 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200235 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200237 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200240 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200242 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200245 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200247 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200251 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200254 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200257 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200259 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200262 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:46:29.201140 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200264 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200267 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200269 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200272 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200275 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200278 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200281 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200283 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200286 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200289 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200291 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200294 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200296 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200299 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200302 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200305 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200307 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200309 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200312 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200315 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:46:29.201628 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200317 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200320 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200322 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200324 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200327 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200729 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200735 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200738 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200741 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200744 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200747 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200750 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200753 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200755 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200758 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200760 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200763 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200765 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200768 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:46:29.202154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200770 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200773 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200776 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200779 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200781 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200784 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200787 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200790 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200792 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200794 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200798 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200800 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200803 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200806 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200808 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200811 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200815 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200818 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200821 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200823 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:46:29.202624 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200826 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200829 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200831 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200834 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200837 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200839 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200842 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200844 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200847 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200849 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200852 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200854 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200857 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200859 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200862 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200864 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200867 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200869 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200871 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200874 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:46:29.203127 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200877 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200879 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200882 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200884 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200887 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200889 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200892 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200894 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200897 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200899 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200902 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200904 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200907 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200910 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200912 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200915 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200917 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200920 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200924 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200928 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:46:29.203656 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200931 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200934 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200937 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200941 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200943 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200946 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200949 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200952 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200955 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200958 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200961 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.200964 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201043 2578 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201052 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201064 2578 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201071 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201077 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201080 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201085 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201089 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201092 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 15:46:29.204146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201096 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201099 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201102 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201106 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201109 2578 flags.go:64] FLAG: --cgroup-root="" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201112 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201115 2578 flags.go:64] FLAG: --client-ca-file="" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201118 2578 flags.go:64] FLAG: --cloud-config="" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201121 2578 flags.go:64] FLAG: --cloud-provider="external" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201124 2578 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201130 2578 flags.go:64] FLAG: --cluster-domain="" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201133 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201136 2578 flags.go:64] FLAG: --config-dir="" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201139 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201143 2578 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201147 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201150 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201154 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201157 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201160 2578 flags.go:64] FLAG: --contention-profiling="false" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201163 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201166 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201169 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201172 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201180 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 15:46:29.204678 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201185 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201189 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201192 2578 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201195 2578 flags.go:64] FLAG: --enable-server="true" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201198 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201202 2578 flags.go:64] FLAG: --event-burst="100" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201205 2578 flags.go:64] FLAG: --event-qps="50" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201208 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201211 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201214 2578 flags.go:64] FLAG: --eviction-hard="" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201219 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201222 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201225 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201228 2578 flags.go:64] FLAG: --eviction-soft="" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201231 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201234 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201237 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201240 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201243 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201246 2578 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201249 2578 flags.go:64] FLAG: --feature-gates="" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201253 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201256 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201259 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201262 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201265 2578 flags.go:64] FLAG: --healthz-port="10248" Jun 02 15:46:29.205275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201268 2578 flags.go:64] FLAG: --help="false" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201271 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201275 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201278 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201281 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201284 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201290 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201293 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201295 2578 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201298 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201301 2578 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201304 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201308 2578 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201310 2578 flags.go:64] FLAG: --kube-reserved="" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201313 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201316 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201320 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201337 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201341 2578 flags.go:64] FLAG: --lock-file="" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201344 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201348 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201351 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201356 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 15:46:29.205911 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201360 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201377 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201381 2578 flags.go:64] FLAG: --logging-format="text" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201384 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201388 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201391 2578 flags.go:64] FLAG: --manifest-url="" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201394 2578 flags.go:64] FLAG: --manifest-url-header="" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201398 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201401 2578 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201405 2578 flags.go:64] FLAG: --max-pods="110" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201408 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201411 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201414 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201417 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201420 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201425 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201429 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201437 2578 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201440 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201443 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201447 2578 flags.go:64] FLAG: --pod-cidr="" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201450 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201456 2578 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201459 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 15:46:29.206489 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201462 2578 flags.go:64] FLAG: --pods-per-core="0" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201466 2578 flags.go:64] FLAG: --port="10250" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201470 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201473 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08c72226a0371da47" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201476 2578 flags.go:64] FLAG: --qos-reserved="" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201479 2578 flags.go:64] FLAG: --read-only-port="10255" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201482 2578 flags.go:64] FLAG: --register-node="true" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201485 2578 flags.go:64] FLAG: --register-schedulable="true" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201488 2578 flags.go:64] FLAG: --register-with-taints="" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201492 2578 flags.go:64] FLAG: --registry-burst="10" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201495 2578 flags.go:64] FLAG: --registry-qps="5" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201497 2578 flags.go:64] FLAG: --reserved-cpus="" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201500 2578 flags.go:64] FLAG: --reserved-memory="" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201504 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201507 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201510 2578 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201513 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201516 2578 flags.go:64] FLAG: --runonce="false" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201519 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201522 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201525 2578 flags.go:64] FLAG: --seccomp-default="false" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201529 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201532 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201535 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201539 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201543 2578 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 15:46:29.207115 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201546 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201549 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201551 2578 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201554 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201557 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201560 2578 flags.go:64] FLAG: --system-cgroups="" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201563 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201570 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201573 2578 flags.go:64] FLAG: --tls-cert-file="" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201575 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201580 2578 flags.go:64] FLAG: --tls-min-version="" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201583 2578 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201586 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201589 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201592 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201595 2578 flags.go:64] FLAG: --v="2" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201600 2578 flags.go:64] FLAG: --version="false" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201603 2578 flags.go:64] FLAG: --vmodule="" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201608 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.201611 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201706 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201710 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201713 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201717 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:46:29.208156 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201719 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201722 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201724 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201727 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201730 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201732 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201736 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201739 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201742 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201745 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201749 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201753 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201756 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201759 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201762 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201766 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201768 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201771 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201774 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:46:29.209180 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201777 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201780 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201782 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201785 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201787 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201790 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201794 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201798 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201800 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201803 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201806 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201810 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201814 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201818 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201821 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201825 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201829 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201833 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201835 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:46:29.210223 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201840 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201843 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201845 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201850 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201853 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201856 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201858 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201861 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201864 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201868 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201871 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201873 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201876 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201879 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201881 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201884 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201887 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201889 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201892 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201895 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:46:29.211081 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201897 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201900 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201903 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201905 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201908 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201910 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201913 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201916 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201918 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201921 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201923 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201926 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201931 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201934 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201937 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201939 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201942 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201945 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201948 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201950 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201952 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:46:29.211837 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201956 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201959 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.201962 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.202523 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.210101 2578 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.210285 2578 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210383 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210393 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210399 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210404 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210408 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210413 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210417 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210421 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210429 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:46:29.212467 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210436 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210441 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210445 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210450 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210455 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210459 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210464 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210468 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210472 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210476 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210482 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210487 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210491 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210495 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210499 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210503 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210508 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210511 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210516 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210520 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:46:29.212986 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210525 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210529 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210546 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210551 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210555 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210560 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210563 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210568 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210572 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210576 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210581 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210585 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210589 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210596 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210603 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210608 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210612 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210618 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210622 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:46:29.213877 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210626 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210630 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210634 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210639 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210643 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210648 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210652 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210656 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210660 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210665 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210669 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210673 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210677 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210682 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210686 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210691 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210703 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210708 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210712 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210716 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:46:29.214677 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210720 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210725 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210729 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210734 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210738 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210742 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210746 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210750 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210754 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210758 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210763 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210767 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210771 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210775 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210779 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210783 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210787 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:46:29.215279 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210791 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.210799 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.210998 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211009 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211016 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211021 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211025 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211030 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211034 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211038 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211044 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211048 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211054 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211058 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211062 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:46:29.215782 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211067 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211071 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211075 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211079 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211083 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211087 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211091 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211095 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211099 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211103 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211107 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211111 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211115 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211120 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211124 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211129 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211133 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211137 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211142 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211147 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:46:29.216218 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211151 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211155 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211162 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211168 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211172 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211176 2578 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211181 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211187 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211191 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211195 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211200 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211205 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211210 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211214 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211218 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211222 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211227 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211231 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211235 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211239 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:46:29.216820 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211243 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211247 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211252 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211256 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211260 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211264 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211268 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211272 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211276 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211281 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211285 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211290 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211294 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211298 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211302 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211315 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211320 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211324 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211329 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211333 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:46:29.217321 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211336 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211341 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211345 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211350 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211355 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211359 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211381 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211386 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211390 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211394 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211398 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211403 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:29.211407 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.211415 2578 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.212130 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 15:46:29.217852 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.215131 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 15:46:29.218226 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.216101 2578 server.go:1019] "Starting client certificate rotation" Jun 02 15:46:29.218226 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.216207 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 15:46:29.218226 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.216241 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 15:46:29.240835 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.240803 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 15:46:29.245559 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.245135 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 15:46:29.260613 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.260589 2578 log.go:25] "Validated CRI v1 runtime API" Jun 02 15:46:29.265744 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.265725 2578 log.go:25] "Validated CRI v1 image API" Jun 02 15:46:29.266951 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.266928 2578 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 15:46:29.270827 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.270802 2578 fs.go:135] Filesystem UUIDs: map[63353da6-cf1c-41d1-b47e-9af9d6b53cf5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a7c68213-3892-4cc6-ad14-da461e7ea4a4:/dev/nvme0n1p4] Jun 02 15:46:29.270936 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.270824 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 15:46:29.271409 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.271390 2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 15:46:29.275380 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.275221 2578 manager.go:217] Machine: {Timestamp:2026-06-02 15:46:29.274726216 +0000 UTC m=+0.404062430 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094931 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2de2960aa09cb8b5cb4fa42c68e9ba SystemUUID:ec2de296-0aa0-9cb8-b5cb-4fa42c68e9ba BootID:d8ab51d8-7735-4dfb-841f-7f0fba4aba3b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bb:0a:0e:2a:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bb:0a:0e:2a:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:37:5d:77:3e:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 15:46:29.275380 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.275349 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 15:46:29.275566 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.275547 2578 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 15:46:29.277664 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.277637 2578 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 15:46:29.277806 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.277665 2578 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 15:46:29.277859 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.277816 2578 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 15:46:29.277859 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.277825 2578 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 15:46:29.277859 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.277833 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 15:46:29.277859 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.277848 2578 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 15:46:29.279132 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.279120 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 02 15:46:29.279270 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.279261 2578 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 15:46:29.281735 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.281723 2578 kubelet.go:491] "Attempting to sync node with API server" Jun 02 15:46:29.281782 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.281739 2578 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 15:46:29.282414 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.282405 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 15:46:29.282447 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.282419 2578 kubelet.go:397] "Adding apiserver pod source" Jun 02 15:46:29.282447 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.282427 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 15:46:29.283466 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.283453 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 15:46:29.283513 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.283473 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 15:46:29.287592 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.287381 2578 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 15:46:29.289768 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.289750 2578 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 15:46:29.291919 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291899 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 15:46:29.291919 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291922 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291931 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291939 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291951 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291961 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291969 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291976 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291987 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.291996 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.292015 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 15:46:29.292070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.292024 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 15:46:29.292856 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.292826 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 15:46:29.292856 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.292842 2578 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 15:46:29.294441 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.294410 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 15:46:29.294441 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.294409 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 15:46:29.296394 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.296359 2578 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 15:46:29.296472 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.296424 2578 server.go:1295] "Started kubelet" Jun 02 15:46:29.296531 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.296485 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 15:46:29.296965 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.296917 2578 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 15:46:29.297057 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.296993 2578 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 15:46:29.297245 ip-10-0-128-123 systemd[1]: Started Kubernetes Kubelet. Jun 02 15:46:29.297949 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.297928 2578 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 15:46:29.299110 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.299098 2578 server.go:317] "Adding debug handlers to kubelet server" Jun 02 15:46:29.300134 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.300115 2578 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 15:46:29.304007 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.303985 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 15:46:29.304504 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.304488 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 15:46:29.304581 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.304559 2578 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 15:46:29.304581 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.304573 2578 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 15:46:29.304748 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.304732 2578 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 15:46:29.304870 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.304826 2578 reconstruct.go:97] "Volume reconstruction finished" Jun 02 15:46:29.304870 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.304858 2578 reconciler.go:26] "Reconciler: start to sync state" Jun 02 15:46:29.305102 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.305082 2578 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 15:46:29.305325 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.305306 2578 factory.go:55] Registering systemd factory Jun 02 15:46:29.305429 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.305327 2578 factory.go:223] Registration of the systemd container factory successfully Jun 02 15:46:29.305614 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.305589 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:29.305675 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.305663 2578 factory.go:153] Registering CRI-O factory Jun 02 15:46:29.305675 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.305674 2578 factory.go:223] Registration of the crio container factory successfully Jun 02 15:46:29.305763 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.305725 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 15:46:29.305763 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.305746 2578 factory.go:103] Registering Raw factory Jun 02 15:46:29.305763 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.305761 2578 manager.go:1196] Started watching for new ooms in manager Jun 02 15:46:29.306123 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.306110 2578 manager.go:319] Starting recovery of all containers Jun 02 15:46:29.306183 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.306146 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 15:46:29.306604 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.305458 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-123.ec2.internal.18b54e2e7eb8a431 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-123.ec2.internal,UID:ip-10-0-128-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-123.ec2.internal,},FirstTimestamp:2026-06-02 15:46:29.296391217 +0000 UTC m=+0.425727444,LastTimestamp:2026-06-02 15:46:29.296391217 +0000 UTC m=+0.425727444,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-123.ec2.internal,}" Jun 02 15:46:29.314179 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.314144 2578 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 15:46:29.317569 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.317546 2578 manager.go:324] Recovery completed Jun 02 15:46:29.321894 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.321868 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:46:29.323747 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.323727 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:46:29.323805 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.323763 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:46:29.323805 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.323777 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:46:29.324323 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.324307 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 15:46:29.324323 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.324321 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 15:46:29.324422 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.324341 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 02 15:46:29.326282 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.326268 2578 policy_none.go:49] "None policy: Start" Jun 02 15:46:29.326282 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.326284 2578 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 15:46:29.326430 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.326294 2578 state_mem.go:35] "Initializing new in-memory state store" Jun 02 15:46:29.326430 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.326293 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-123.ec2.internal.18b54e2e805a0f11 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-123.ec2.internal,UID:ip-10-0-128-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-123.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-123.ec2.internal,},FirstTimestamp:2026-06-02 15:46:29.323747089 +0000 UTC m=+0.453083306,LastTimestamp:2026-06-02 15:46:29.323747089 +0000 UTC m=+0.453083306,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-123.ec2.internal,}" Jun 02 15:46:29.337630 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.337531 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-123.ec2.internal.18b54e2e805a68d9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-123.ec2.internal,UID:ip-10-0-128-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-123.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-123.ec2.internal,},FirstTimestamp:2026-06-02 15:46:29.323770073 +0000 UTC m=+0.453106291,LastTimestamp:2026-06-02 15:46:29.323770073 +0000 UTC m=+0.453106291,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-123.ec2.internal,}" Jun 02 15:46:29.337771 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.337749 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dnm2w" Jun 02 15:46:29.347006 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.346983 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dnm2w" Jun 02 15:46:29.350198 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.350125 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-123.ec2.internal.18b54e2e805a94f8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-123.ec2.internal,UID:ip-10-0-128-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-123.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-123.ec2.internal,},FirstTimestamp:2026-06-02 15:46:29.323781368 +0000 UTC m=+0.453117585,LastTimestamp:2026-06-02 15:46:29.323781368 +0000 UTC m=+0.453117585,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-123.ec2.internal,}" Jun 02 15:46:29.364128 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.364108 2578 manager.go:341] "Starting Device Plugin manager" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.364211 2578 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.364225 2578 server.go:85] "Starting device plugin registration server" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.364536 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.364559 2578 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.364646 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.364743 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.364755 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.365650 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 15:46:29.375335 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.365693 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:29.408972 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.408933 2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 15:46:29.410093 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.410080 2578 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 15:46:29.410152 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.410108 2578 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 15:46:29.410152 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.410129 2578 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 15:46:29.410152 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.410136 2578 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 15:46:29.410292 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.410241 2578 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 15:46:29.413059 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.413030 2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:46:29.465237 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.465172 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:46:29.466186 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.466168 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:46:29.466250 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.466202 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:46:29.466250 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.466212 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:46:29.466250 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.466235 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.476025 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.476004 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.476077 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.476031 2578 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-123.ec2.internal\": node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:29.493210 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.493180 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:29.510379 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.510347 2578 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal"] Jun 02 15:46:29.510526 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.510452 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:46:29.511457 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.511440 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:46:29.511531 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.511472 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:46:29.511531 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.511487 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:46:29.512705 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.512691 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:46:29.512900 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.512885 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.512939 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.512917 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:46:29.513490 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.513458 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:46:29.513555 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.513498 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:46:29.513555 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.513508 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:46:29.513555 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.513519 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:46:29.513555 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.513539 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:46:29.513555 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.513549 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:46:29.514497 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.514482 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.514553 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.514507 2578 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:46:29.515225 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.515198 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:46:29.515225 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.515221 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:46:29.515348 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.515245 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:46:29.533008 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.532983 2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-123.ec2.internal\" not found" node="ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.537264 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.537249 2578 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-123.ec2.internal\" not found" node="ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.593311 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.593283 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:29.693640 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.693600 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:29.705885 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.705855 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/994b7340992241a93e28eb8a5fde6ba9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"994b7340992241a93e28eb8a5fde6ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.705990 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.705893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/994b7340992241a93e28eb8a5fde6ba9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"994b7340992241a93e28eb8a5fde6ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.705990 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.705950 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131ee351bb55ad6be954235ac6b9f987-config\") pod \"kube-apiserver-proxy-ip-10-0-128-123.ec2.internal\" (UID: \"131ee351bb55ad6be954235ac6b9f987\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.794391 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.794293 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:29.806677 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.806648 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/994b7340992241a93e28eb8a5fde6ba9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"994b7340992241a93e28eb8a5fde6ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.806743 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.806684 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131ee351bb55ad6be954235ac6b9f987-config\") pod \"kube-apiserver-proxy-ip-10-0-128-123.ec2.internal\" (UID: \"131ee351bb55ad6be954235ac6b9f987\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.806743 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.806701 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/994b7340992241a93e28eb8a5fde6ba9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"994b7340992241a93e28eb8a5fde6ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.806826 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.806738 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/994b7340992241a93e28eb8a5fde6ba9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"994b7340992241a93e28eb8a5fde6ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.806826 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.806750 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/994b7340992241a93e28eb8a5fde6ba9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal\" (UID: \"994b7340992241a93e28eb8a5fde6ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.806826 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.806752 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/131ee351bb55ad6be954235ac6b9f987-config\") pod \"kube-apiserver-proxy-ip-10-0-128-123.ec2.internal\" (UID: \"131ee351bb55ad6be954235ac6b9f987\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.834773 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.834742 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.840335 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:29.840314 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" Jun 02 15:46:29.894972 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.894929 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:29.995480 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:29.995447 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:30.096030 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:30.095945 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:30.196562 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:30.196526 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:30.215996 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.215967 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 15:46:30.216135 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.216122 2578 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 15:46:30.297612 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:30.297577 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:30.304352 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.304326 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 15:46:30.318897 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:30.318727 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131ee351bb55ad6be954235ac6b9f987.slice/crio-90e0408e5a7a99f6afd50ad1716a32e367e842eed351bc68949459c18ff671d8 WatchSource:0}: Error finding container 90e0408e5a7a99f6afd50ad1716a32e367e842eed351bc68949459c18ff671d8: Status 404 returned error can't find the container with id 90e0408e5a7a99f6afd50ad1716a32e367e842eed351bc68949459c18ff671d8 Jun 02 15:46:30.319079 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:30.319059 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod994b7340992241a93e28eb8a5fde6ba9.slice/crio-7601fe8c9a7dbfb907454d7bdcc0b17dcfa306376fd331a9c50008439dd67ab1 WatchSource:0}: Error finding container 7601fe8c9a7dbfb907454d7bdcc0b17dcfa306376fd331a9c50008439dd67ab1: Status 404 returned error can't find the container with id 7601fe8c9a7dbfb907454d7bdcc0b17dcfa306376fd331a9c50008439dd67ab1 Jun 02 15:46:30.319301 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.319281 2578 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 15:46:30.324535 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.324521 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 15:46:30.348955 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.348880 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 15:41:29 +0000 UTC" deadline="2027-11-15 07:53:07.486470117 +0000 UTC" Jun 02 15:46:30.348955 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.348910 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12736h6m37.137562781s" Jun 02 15:46:30.389567 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.389541 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-prpcg" Jun 02 15:46:30.398634 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:30.398612 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:30.399996 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.399984 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-prpcg" Jun 02 15:46:30.414012 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.413965 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" event={"ID":"131ee351bb55ad6be954235ac6b9f987","Type":"ContainerStarted","Data":"90e0408e5a7a99f6afd50ad1716a32e367e842eed351bc68949459c18ff671d8"} Jun 02 15:46:30.415272 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.415251 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" event={"ID":"994b7340992241a93e28eb8a5fde6ba9","Type":"ContainerStarted","Data":"7601fe8c9a7dbfb907454d7bdcc0b17dcfa306376fd331a9c50008439dd67ab1"} Jun 02 15:46:30.499471 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:30.499441 2578 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-123.ec2.internal\" not found" Jun 02 15:46:30.546361 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.545403 2578 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:46:30.605643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.605568 2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" Jun 02 15:46:30.621275 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.621253 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 15:46:30.622914 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.622901 2578 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" Jun 02 15:46:30.629171 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.629155 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 15:46:30.783153 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.783121 2578 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:46:30.846510 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:30.846478 2578 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:46:31.283538 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.283501 2578 apiserver.go:52] "Watching apiserver" Jun 02 15:46:31.293047 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.293016 2578 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 15:46:31.294403 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.294346 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-thdlh","openshift-dns/node-resolver-q4c6g","openshift-image-registry/node-ca-5ptj6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal","openshift-network-diagnostics/network-check-target-94lpx","openshift-ovn-kubernetes/ovnkube-node-cnkrg","kube-system/konnectivity-agent-lvvg8","kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt","openshift-multus/multus-additional-cni-plugins-qlzs5","openshift-multus/multus-zc6zt","openshift-multus/network-metrics-daemon-7wvwq","openshift-network-operator/iptables-alerter-wpk7x"] Jun 02 15:46:31.296091 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.296060 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.297233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.297211 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.298407 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.298387 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 15:46:31.298514 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.298432 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.298587 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.298523 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 15:46:31.298679 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.298655 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 15:46:31.298850 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.298833 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 15:46:31.298960 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.298944 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 15:46:31.299324 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.299132 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 15:46:31.299324 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.299131 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mxkrk\"" Jun 02 15:46:31.299324 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.299226 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7nkv9\"" Jun 02 15:46:31.299324 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.299266 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 15:46:31.299941 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.299912 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:31.300031 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:31.299993 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:31.300718 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.300695 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 15:46:31.300949 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.300935 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 15:46:31.301167 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.301142 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 15:46:31.301167 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.301161 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lqc85\"" Jun 02 15:46:31.302617 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.302592 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.304569 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.304340 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:31.305434 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.305023 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 15:46:31.305434 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.305071 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 15:46:31.306095 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.305930 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 15:46:31.306229 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.306212 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 15:46:31.306635 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.306549 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 15:46:31.306901 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.306814 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cmx6c\"" Jun 02 15:46:31.307348 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.307137 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 15:46:31.307348 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.307168 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wh6fq\"" Jun 02 15:46:31.307348 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.307330 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 15:46:31.307547 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.307495 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 15:46:31.308174 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.308051 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.308174 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.308171 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.310522 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.310499 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:46:31.310604 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.310543 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.311170 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.310747 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vvjdv\"" Jun 02 15:46:31.311170 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.310928 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 15:46:31.311170 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.310962 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 15:46:31.311170 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.311010 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 15:46:31.311170 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.311064 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f7rsj\"" Jun 02 15:46:31.311170 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.311084 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 15:46:31.312144 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.312125 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:31.312243 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:31.312217 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:31.312308 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.312260 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 15:46:31.312653 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.312632 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mnqvv\"" Jun 02 15:46:31.313458 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.313435 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.314754 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-run\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.314848 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-ovn\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.314848 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d794b47c-d098-448e-926e-1e81d554e31f-host\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.314848 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d794b47c-d098-448e-926e-1e81d554e31f-serviceca\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.314998 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-tmp-dir\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.314998 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-run-netns\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.314998 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-systemd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.314998 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovnkube-script-lib\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.314998 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.314969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhssd\" (UniqueName: \"kubernetes.io/projected/6d0b6596-6b7d-4228-85d3-95620e2804a7-kube-api-access-dhssd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.315233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315014 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-sys-fs\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.315233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm7nm\" (UniqueName: \"kubernetes.io/projected/d794b47c-d098-448e-926e-1e81d554e31f-kube-api-access-nm7nm\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.315233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315091 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f2fd357-da07-4b2a-a98b-29ea324b4971-konnectivity-ca\") pod \"konnectivity-agent-lvvg8\" (UID: \"6f2fd357-da07-4b2a-a98b-29ea324b4971\") " pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:31.315233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315114 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-slash\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.315233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2wjs\" (UniqueName: \"kubernetes.io/projected/893b281a-ffc7-4fdd-b162-738f9f32da0c-kube-api-access-s2wjs\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.315233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315163 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc82l\" (UniqueName: \"kubernetes.io/projected/e430bfcf-7490-4927-b6cc-f89f9d081f39-kube-api-access-kc82l\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.315233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-var-lib-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.315233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovnkube-config\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwpvz\" (UniqueName: \"kubernetes.io/projected/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-kube-api-access-fwpvz\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-kubelet\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-modprobe-d\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315338 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315357 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hz8x6\"" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315427 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-log-socket\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f2fd357-da07-4b2a-a98b-29ea324b4971-agent-certs\") pod \"konnectivity-agent-lvvg8\" (UID: \"6f2fd357-da07-4b2a-a98b-29ea324b4971\") " pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315602 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:46:31.315643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-node-log\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x789t\" (UniqueName: \"kubernetes.io/projected/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-kube-api-access-x789t\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysconfig\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315735 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-var-lib-kubelet\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-system-cni-dir\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315819 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-os-release\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-etc-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-cni-bin\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.315974 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-lib-modules\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316054 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.316141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316121 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-hosts-file\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-env-overrides\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316177 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-device-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/893b281a-ffc7-4fdd-b162-738f9f32da0c-tmp\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316229 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316254 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-cni-netd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysctl-d\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316390 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316413 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-socket-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-kubernetes\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-systemd\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316487 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-host\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-tuned\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-cnibin\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.316867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316572 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovn-node-metrics-cert\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.317614 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-sys\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.317614 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316656 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.317614 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-registration-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.317614 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316709 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-systemd-units\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.317614 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.316732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysctl-conf\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.401219 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.401185 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 15:41:30 +0000 UTC" deadline="2028-01-22 01:57:15.591181807 +0000 UTC" Jun 02 15:46:31.401219 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.401213 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14362h10m44.189972274s" Jun 02 15:46:31.406773 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.406749 2578 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 15:46:31.417709 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417676 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-multus-certs\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.417867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417740 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f2fd357-da07-4b2a-a98b-29ea324b4971-agent-certs\") pod \"konnectivity-agent-lvvg8\" (UID: \"6f2fd357-da07-4b2a-a98b-29ea324b4971\") " pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:31.417867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417770 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.417867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417808 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-node-log\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.417867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417833 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.417867 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417856 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x789t\" (UniqueName: \"kubernetes.io/projected/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-kube-api-access-x789t\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/010e3146-6f24-4c4f-853e-899ae1a2fe39-host-slash\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417905 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysconfig\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417925 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-var-lib-kubelet\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417947 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-system-cni-dir\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417969 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-os-release\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.417993 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-etc-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418017 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418042 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-cni-bin\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418066 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418091 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-lib-modules\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.418121 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418116 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418139 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-hosts-file\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418169 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418240 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-system-cni-dir\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-node-log\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418345 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418392 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418490 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysconfig\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418552 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-cni-bin\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418601 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-var-lib-kubelet\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.418644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418613 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-env-overrides\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418653 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-os-release\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418603 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-etc-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418683 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-hosts-file\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418701 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418720 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418175 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-env-overrides\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418793 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-device-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418816 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-lib-modules\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418826 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2ds2\" (UniqueName: \"kubernetes.io/projected/877796b1-2b26-4c87-903a-1235f441e838-kube-api-access-g2ds2\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-etc-kubernetes\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418868 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-device-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418879 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/893b281a-ffc7-4fdd-b162-738f9f32da0c-tmp\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418915 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418942 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418942 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418978 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.419099 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.418988 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-cni-netd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419013 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-cni-netd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-socket-dir-parent\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419051 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-conf-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419078 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419109 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/010e3146-6f24-4c4f-853e-899ae1a2fe39-iptables-alerter-script\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419144 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysctl-d\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419172 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419201 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-socket-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fhnn\" (UniqueName: \"kubernetes.io/projected/010e3146-6f24-4c4f-853e-899ae1a2fe39-kube-api-access-8fhnn\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419237 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419248 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-kubernetes\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419271 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysctl-d\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419273 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-systemd\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419318 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-host\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-systemd\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419343 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-tuned\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.419883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419383 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-cnibin\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419409 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovn-node-metrics-cert\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419412 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-kubernetes\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419382 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-socket-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419437 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-host\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-cnibin\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419490 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e430bfcf-7490-4927-b6cc-f89f9d081f39-cnibin\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419505 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-daemon-config\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419536 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-sys\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419566 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419602 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-registration-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-os-release\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419663 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-systemd-units\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419669 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6w86\" (UniqueName: \"kubernetes.io/projected/7a43ecc6-b5eb-435b-9737-ee046a1013ab-kube-api-access-s6w86\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419703 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e430bfcf-7490-4927-b6cc-f89f9d081f39-cni-binary-copy\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419714 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysctl-conf\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.420726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419639 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-sys\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419726 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-registration-dir\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-cni-multus\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419766 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-systemd-units\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419770 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-run\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419797 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-ovn\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419815 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-run\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419821 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-kubelet\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419832 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-sysctl-conf\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419847 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d794b47c-d098-448e-926e-1e81d554e31f-host\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419854 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-ovn\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419869 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d794b47c-d098-448e-926e-1e81d554e31f-serviceca\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419896 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-tmp-dir\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419900 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d794b47c-d098-448e-926e-1e81d554e31f-host\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419921 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-run-netns\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419946 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-systemd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419969 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovnkube-script-lib\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.419997 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dhssd\" (UniqueName: \"kubernetes.io/projected/6d0b6596-6b7d-4228-85d3-95620e2804a7-kube-api-access-dhssd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.421491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420021 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-sys-fs\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420046 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nm7nm\" (UniqueName: \"kubernetes.io/projected/d794b47c-d098-448e-926e-1e81d554e31f-kube-api-access-nm7nm\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420070 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f2fd357-da07-4b2a-a98b-29ea324b4971-konnectivity-ca\") pod \"konnectivity-agent-lvvg8\" (UID: \"6f2fd357-da07-4b2a-a98b-29ea324b4971\") " pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420095 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-slash\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-system-cni-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420149 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-cni-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a43ecc6-b5eb-435b-9737-ee046a1013ab-cni-binary-copy\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420176 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-tmp-dir\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420198 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2wjs\" (UniqueName: \"kubernetes.io/projected/893b281a-ffc7-4fdd-b162-738f9f32da0c-kube-api-access-s2wjs\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420228 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kc82l\" (UniqueName: \"kubernetes.io/projected/e430bfcf-7490-4927-b6cc-f89f9d081f39-kube-api-access-kc82l\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420254 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-var-lib-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420279 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovnkube-config\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-netns\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420332 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fwpvz\" (UniqueName: \"kubernetes.io/projected/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-kube-api-access-fwpvz\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420358 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-kubelet\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-hostroot\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420439 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-modprobe-d\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.422283 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420460 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-run-netns\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d794b47c-d098-448e-926e-1e81d554e31f-serviceca\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420516 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-log-socket\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420550 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-run-systemd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420463 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-log-socket\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-k8s-cni-cncf-io\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-cni-bin\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420728 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-var-lib-openvswitch\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420769 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-slash\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.420926 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f2fd357-da07-4b2a-a98b-29ea324b4971-konnectivity-ca\") pod \"konnectivity-agent-lvvg8\" (UID: \"6f2fd357-da07-4b2a-a98b-29ea324b4971\") " pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.421202 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d0b6596-6b7d-4228-85d3-95620e2804a7-host-kubelet\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.421252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-sys-fs\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.421272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-modprobe-d\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.421324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovnkube-config\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.421583 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovnkube-script-lib\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.422343 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f2fd357-da07-4b2a-a98b-29ea324b4971-agent-certs\") pod \"konnectivity-agent-lvvg8\" (UID: \"6f2fd357-da07-4b2a-a98b-29ea324b4971\") " pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:31.423055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.422464 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d0b6596-6b7d-4228-85d3-95620e2804a7-ovn-node-metrics-cert\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.423834 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.423467 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/893b281a-ffc7-4fdd-b162-738f9f32da0c-etc-tuned\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.423834 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.423698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/893b281a-ffc7-4fdd-b162-738f9f32da0c-tmp\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.426508 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.426484 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x789t\" (UniqueName: \"kubernetes.io/projected/9c7ecca4-9300-4a1c-8f78-b999fe91d8e5-kube-api-access-x789t\") pod \"aws-ebs-csi-driver-node-mgdbt\" (UID: \"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.428308 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:31.428286 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:31.428464 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:31.428315 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:31.428464 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:31.428328 2578 projected.go:194] Error preparing data for projected volume kube-api-access-w9tqr for pod openshift-network-diagnostics/network-check-target-94lpx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:31.428464 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:31.428455 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr podName:3a94647c-a5b5-4c9d-a13d-d6c824a3a79e nodeName:}" failed. No retries permitted until 2026-06-02 15:46:31.928396672 +0000 UTC m=+3.057732895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w9tqr" (UniqueName: "kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr") pod "network-check-target-94lpx" (UID: "3a94647c-a5b5-4c9d-a13d-d6c824a3a79e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:31.432328 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.432305 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc82l\" (UniqueName: \"kubernetes.io/projected/e430bfcf-7490-4927-b6cc-f89f9d081f39-kube-api-access-kc82l\") pod \"multus-additional-cni-plugins-qlzs5\" (UID: \"e430bfcf-7490-4927-b6cc-f89f9d081f39\") " pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.432985 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.432960 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwpvz\" (UniqueName: \"kubernetes.io/projected/2de345a5-85e0-422e-8497-e9d7c2fbdcc4-kube-api-access-fwpvz\") pod \"node-resolver-q4c6g\" (UID: \"2de345a5-85e0-422e-8497-e9d7c2fbdcc4\") " pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.434053 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.434032 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2wjs\" (UniqueName: \"kubernetes.io/projected/893b281a-ffc7-4fdd-b162-738f9f32da0c-kube-api-access-s2wjs\") pod \"tuned-thdlh\" (UID: \"893b281a-ffc7-4fdd-b162-738f9f32da0c\") " pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.434164 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.434117 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhssd\" (UniqueName: \"kubernetes.io/projected/6d0b6596-6b7d-4228-85d3-95620e2804a7-kube-api-access-dhssd\") pod \"ovnkube-node-cnkrg\" (UID: \"6d0b6596-6b7d-4228-85d3-95620e2804a7\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.434228 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.434187 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm7nm\" (UniqueName: \"kubernetes.io/projected/d794b47c-d098-448e-926e-1e81d554e31f-kube-api-access-nm7nm\") pod \"node-ca-5ptj6\" (UID: \"d794b47c-d098-448e-926e-1e81d554e31f\") " pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.521826 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521796 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-system-cni-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521839 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-cni-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521866 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a43ecc6-b5eb-435b-9737-ee046a1013ab-cni-binary-copy\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521894 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-netns\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521910 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-hostroot\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521938 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-system-cni-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521953 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-cni-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521946 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521992 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-netns\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.521999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-hostroot\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522005 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-k8s-cni-cncf-io\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522042 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-cni-bin\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522051 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-k8s-cni-cncf-io\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522070 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-multus-certs\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522088 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-cni-bin\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:31.522095 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522091 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/010e3146-6f24-4c4f-853e-899ae1a2fe39-host-slash\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522115 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/010e3146-6f24-4c4f-853e-899ae1a2fe39-host-slash\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522147 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g2ds2\" (UniqueName: \"kubernetes.io/projected/877796b1-2b26-4c87-903a-1235f441e838-kube-api-access-g2ds2\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:31.522173 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs podName:877796b1-2b26-4c87-903a-1235f441e838 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:32.022150488 +0000 UTC m=+3.151486715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs") pod "network-metrics-daemon-7wvwq" (UID: "877796b1-2b26-4c87-903a-1235f441e838") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522178 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-run-multus-certs\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522199 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-etc-kubernetes\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522240 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-socket-dir-parent\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522263 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-conf-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522289 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/010e3146-6f24-4c4f-853e-899ae1a2fe39-iptables-alerter-script\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522307 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-etc-kubernetes\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522329 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-conf-dir\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.522483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522331 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-socket-dir-parent\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522355 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8fhnn\" (UniqueName: \"kubernetes.io/projected/010e3146-6f24-4c4f-853e-899ae1a2fe39-kube-api-access-8fhnn\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522404 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-cnibin\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522429 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-daemon-config\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522458 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-os-release\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522483 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s6w86\" (UniqueName: \"kubernetes.io/projected/7a43ecc6-b5eb-435b-9737-ee046a1013ab-kube-api-access-s6w86\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522507 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-cni-multus\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522509 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-cnibin\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522533 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-kubelet\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522562 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7a43ecc6-b5eb-435b-9737-ee046a1013ab-cni-binary-copy\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522579 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-os-release\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522600 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-kubelet\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522640 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7a43ecc6-b5eb-435b-9737-ee046a1013ab-host-var-lib-cni-multus\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.522861 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/010e3146-6f24-4c4f-853e-899ae1a2fe39-iptables-alerter-script\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.523245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.523047 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7a43ecc6-b5eb-435b-9737-ee046a1013ab-multus-daemon-config\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.529586 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.529562 2578 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:46:31.534224 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.534159 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fhnn\" (UniqueName: \"kubernetes.io/projected/010e3146-6f24-4c4f-853e-899ae1a2fe39-kube-api-access-8fhnn\") pod \"iptables-alerter-wpk7x\" (UID: \"010e3146-6f24-4c4f-853e-899ae1a2fe39\") " pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:31.534565 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.534546 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6w86\" (UniqueName: \"kubernetes.io/projected/7a43ecc6-b5eb-435b-9737-ee046a1013ab-kube-api-access-s6w86\") pod \"multus-zc6zt\" (UID: \"7a43ecc6-b5eb-435b-9737-ee046a1013ab\") " pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.534760 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.534743 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2ds2\" (UniqueName: \"kubernetes.io/projected/877796b1-2b26-4c87-903a-1235f441e838-kube-api-access-g2ds2\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:31.610754 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.610712 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" Jun 02 15:46:31.619451 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.619430 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q4c6g" Jun 02 15:46:31.629111 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.629090 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5ptj6" Jun 02 15:46:31.635740 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.635714 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:31.643393 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.643362 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:31.651668 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.651643 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-thdlh" Jun 02 15:46:31.658272 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.658250 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" Jun 02 15:46:31.664879 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.664857 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zc6zt" Jun 02 15:46:31.671519 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:31.671500 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wpk7x" Jun 02 15:46:32.025330 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.025219 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:32.025330 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.025278 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:32.025554 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:32.025374 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:32.025554 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:32.025393 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:32.025554 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:32.025404 2578 projected.go:194] Error preparing data for projected volume kube-api-access-w9tqr for pod openshift-network-diagnostics/network-check-target-94lpx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:32.025554 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:32.025451 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr podName:3a94647c-a5b5-4c9d-a13d-d6c824a3a79e nodeName:}" failed. No retries permitted until 2026-06-02 15:46:33.025434624 +0000 UTC m=+4.154770830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9tqr" (UniqueName: "kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr") pod "network-check-target-94lpx" (UID: "3a94647c-a5b5-4c9d-a13d-d6c824a3a79e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:32.025554 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:32.025391 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:32.025554 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:32.025520 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs podName:877796b1-2b26-4c87-903a-1235f441e838 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:33.025500683 +0000 UTC m=+4.154836884 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs") pod "network-metrics-daemon-7wvwq" (UID: "877796b1-2b26-4c87-903a-1235f441e838") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:32.063792 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:32.063727 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd794b47c_d098_448e_926e_1e81d554e31f.slice/crio-b8b3da8c6a674a2b7e67bbd6f460351e144f4e944bfa8e48c9948ad9a5dcde4c WatchSource:0}: Error finding container b8b3da8c6a674a2b7e67bbd6f460351e144f4e944bfa8e48c9948ad9a5dcde4c: Status 404 returned error can't find the container with id b8b3da8c6a674a2b7e67bbd6f460351e144f4e944bfa8e48c9948ad9a5dcde4c Jun 02 15:46:32.064873 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:32.064841 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010e3146_6f24_4c4f_853e_899ae1a2fe39.slice/crio-3c70b4f0455b35ee7215b7fbe1c22f75bac5208697825132a38869e4c45ed6c2 WatchSource:0}: Error finding container 3c70b4f0455b35ee7215b7fbe1c22f75bac5208697825132a38869e4c45ed6c2: Status 404 returned error can't find the container with id 3c70b4f0455b35ee7215b7fbe1c22f75bac5208697825132a38869e4c45ed6c2 Jun 02 15:46:32.066021 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:32.065995 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de345a5_85e0_422e_8497_e9d7c2fbdcc4.slice/crio-c31f001ca61c93e05f9e7e5ab9b9555c0a030f90606318c2d914768c18ac4e7a WatchSource:0}: Error finding container c31f001ca61c93e05f9e7e5ab9b9555c0a030f90606318c2d914768c18ac4e7a: Status 404 returned error can't find the container with id c31f001ca61c93e05f9e7e5ab9b9555c0a030f90606318c2d914768c18ac4e7a Jun 02 15:46:32.067670 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:32.067635 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod893b281a_ffc7_4fdd_b162_738f9f32da0c.slice/crio-88cbf9f56ecf383b7dbdfdafd2279c1e665fd9aede788faa2a25dcfc3b672309 WatchSource:0}: Error finding container 88cbf9f56ecf383b7dbdfdafd2279c1e665fd9aede788faa2a25dcfc3b672309: Status 404 returned error can't find the container with id 88cbf9f56ecf383b7dbdfdafd2279c1e665fd9aede788faa2a25dcfc3b672309 Jun 02 15:46:32.073336 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:32.071348 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a43ecc6_b5eb_435b_9737_ee046a1013ab.slice/crio-1bb570e2e4623cf447338274e8cdea2fffc0a0c367d66566c45d073eebb381db WatchSource:0}: Error finding container 1bb570e2e4623cf447338274e8cdea2fffc0a0c367d66566c45d073eebb381db: Status 404 returned error can't find the container with id 1bb570e2e4623cf447338274e8cdea2fffc0a0c367d66566c45d073eebb381db Jun 02 15:46:32.075769 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:32.075643 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode430bfcf_7490_4927_b6cc_f89f9d081f39.slice/crio-4063bdc898ca2829292fc6aac10dbf8367dec711fedc92fa1b30937f1abe00f0 WatchSource:0}: Error finding container 4063bdc898ca2829292fc6aac10dbf8367dec711fedc92fa1b30937f1abe00f0: Status 404 returned error can't find the container with id 4063bdc898ca2829292fc6aac10dbf8367dec711fedc92fa1b30937f1abe00f0 Jun 02 15:46:32.076102 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:32.076079 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7ecca4_9300_4a1c_8f78_b999fe91d8e5.slice/crio-e01fdcea8859903fb7a365c95e4178edbcc997c3ee6a76a988bea80cafa18f8b WatchSource:0}: Error finding container e01fdcea8859903fb7a365c95e4178edbcc997c3ee6a76a988bea80cafa18f8b: Status 404 returned error can't find the container with id e01fdcea8859903fb7a365c95e4178edbcc997c3ee6a76a988bea80cafa18f8b Jun 02 15:46:32.077504 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:46:32.077478 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f2fd357_da07_4b2a_a98b_29ea324b4971.slice/crio-f57fd4c1d0cf63ef6b3e5731a4f6b7effd58805569018d63d2f42a5f81701543 WatchSource:0}: Error finding container f57fd4c1d0cf63ef6b3e5731a4f6b7effd58805569018d63d2f42a5f81701543: Status 404 returned error can't find the container with id f57fd4c1d0cf63ef6b3e5731a4f6b7effd58805569018d63d2f42a5f81701543 Jun 02 15:46:32.402414 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.402348 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 15:41:30 +0000 UTC" deadline="2027-11-26 07:59:55.860762866 +0000 UTC" Jun 02 15:46:32.402414 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.402403 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13000h13m23.458363371s" Jun 02 15:46:32.410675 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.410642 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:32.410675 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.410671 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:32.410935 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:32.410765 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:32.410935 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:32.410899 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:32.419948 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.419896 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lvvg8" event={"ID":"6f2fd357-da07-4b2a-a98b-29ea324b4971","Type":"ContainerStarted","Data":"f57fd4c1d0cf63ef6b3e5731a4f6b7effd58805569018d63d2f42a5f81701543"} Jun 02 15:46:32.421329 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.421289 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerStarted","Data":"4063bdc898ca2829292fc6aac10dbf8367dec711fedc92fa1b30937f1abe00f0"} Jun 02 15:46:32.422419 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.422394 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-thdlh" event={"ID":"893b281a-ffc7-4fdd-b162-738f9f32da0c","Type":"ContainerStarted","Data":"88cbf9f56ecf383b7dbdfdafd2279c1e665fd9aede788faa2a25dcfc3b672309"} Jun 02 15:46:32.423483 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.423461 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q4c6g" event={"ID":"2de345a5-85e0-422e-8497-e9d7c2fbdcc4","Type":"ContainerStarted","Data":"c31f001ca61c93e05f9e7e5ab9b9555c0a030f90606318c2d914768c18ac4e7a"} Jun 02 15:46:32.424481 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.424463 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wpk7x" event={"ID":"010e3146-6f24-4c4f-853e-899ae1a2fe39","Type":"ContainerStarted","Data":"3c70b4f0455b35ee7215b7fbe1c22f75bac5208697825132a38869e4c45ed6c2"} Jun 02 15:46:32.425475 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.425450 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5ptj6" event={"ID":"d794b47c-d098-448e-926e-1e81d554e31f","Type":"ContainerStarted","Data":"b8b3da8c6a674a2b7e67bbd6f460351e144f4e944bfa8e48c9948ad9a5dcde4c"} Jun 02 15:46:32.427234 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.427208 2578 generic.go:358] "Generic (PLEG): container finished" podID="994b7340992241a93e28eb8a5fde6ba9" containerID="57b5370e43fb60808d121aabf50d43d9c4fcc8c0c1b2b80a25d7f51d8241c748" exitCode=0 Jun 02 15:46:32.427333 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.427283 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" event={"ID":"994b7340992241a93e28eb8a5fde6ba9","Type":"ContainerDied","Data":"57b5370e43fb60808d121aabf50d43d9c4fcc8c0c1b2b80a25d7f51d8241c748"} Jun 02 15:46:32.429065 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.429044 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" event={"ID":"131ee351bb55ad6be954235ac6b9f987","Type":"ContainerStarted","Data":"2eead5b03d462c03b054ff946585046c0f1fa7aa4f9b9fa15ebf2470316942a3"} Jun 02 15:46:32.430806 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.430782 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" event={"ID":"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5","Type":"ContainerStarted","Data":"e01fdcea8859903fb7a365c95e4178edbcc997c3ee6a76a988bea80cafa18f8b"} Jun 02 15:46:32.432866 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.432835 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"d0b3b66f32bda8d9a961dea1ace3fa10bc7db6644acdd48b9e92817bbb4563d7"} Jun 02 15:46:32.435833 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:32.435795 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zc6zt" event={"ID":"7a43ecc6-b5eb-435b-9737-ee046a1013ab","Type":"ContainerStarted","Data":"1bb570e2e4623cf447338274e8cdea2fffc0a0c367d66566c45d073eebb381db"} Jun 02 15:46:33.031902 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:33.031862 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:33.032070 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:33.031924 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:33.032070 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:33.032065 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:33.032196 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:33.032083 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:33.032196 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:33.032097 2578 projected.go:194] Error preparing data for projected volume kube-api-access-w9tqr for pod openshift-network-diagnostics/network-check-target-94lpx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:33.032196 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:33.032157 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr podName:3a94647c-a5b5-4c9d-a13d-d6c824a3a79e nodeName:}" failed. No retries permitted until 2026-06-02 15:46:35.032138114 +0000 UTC m=+6.161474328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9tqr" (UniqueName: "kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr") pod "network-check-target-94lpx" (UID: "3a94647c-a5b5-4c9d-a13d-d6c824a3a79e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:33.032669 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:33.032605 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:33.032669 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:33.032661 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs podName:877796b1-2b26-4c87-903a-1235f441e838 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:35.032645256 +0000 UTC m=+6.161981464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs") pod "network-metrics-daemon-7wvwq" (UID: "877796b1-2b26-4c87-903a-1235f441e838") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:33.450347 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:33.450290 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" event={"ID":"994b7340992241a93e28eb8a5fde6ba9","Type":"ContainerStarted","Data":"4f31cf86de8108108dfdacb69c0106e732d1455e7e60355603e64afd04058e3e"} Jun 02 15:46:33.466083 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:33.464850 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-123.ec2.internal" podStartSLOduration=3.464831292 podStartE2EDuration="3.464831292s" podCreationTimestamp="2026-06-02 15:46:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:46:32.456475009 +0000 UTC m=+3.585811233" watchObservedRunningTime="2026-06-02 15:46:33.464831292 +0000 UTC m=+4.594167507" Jun 02 15:46:34.410881 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.410847 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:34.411074 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:34.410973 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:34.411386 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.411341 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:34.411529 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:34.411474 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:34.779633 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.779520 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-123.ec2.internal" podStartSLOduration=4.77950128 podStartE2EDuration="4.77950128s" podCreationTimestamp="2026-06-02 15:46:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:46:33.466064928 +0000 UTC m=+4.595401130" watchObservedRunningTime="2026-06-02 15:46:34.77950128 +0000 UTC m=+5.908837503" Jun 02 15:46:34.780491 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.780468 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-5f5cp"] Jun 02 15:46:34.782108 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.782088 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.782200 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:34.782169 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:34.846611 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.846340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8949689-7df4-4ab2-b219-e6bc3ee79e40-dbus\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.846611 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.846426 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8949689-7df4-4ab2-b219-e6bc3ee79e40-kubelet-config\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.846611 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.846507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.948060 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.947841 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8949689-7df4-4ab2-b219-e6bc3ee79e40-kubelet-config\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.948060 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.947928 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.948060 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.947932 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8949689-7df4-4ab2-b219-e6bc3ee79e40-kubelet-config\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.948060 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.947963 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8949689-7df4-4ab2-b219-e6bc3ee79e40-dbus\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.948060 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:34.948031 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8949689-7df4-4ab2-b219-e6bc3ee79e40-dbus\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:34.948060 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:34.948044 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:34.948482 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:34.948098 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret podName:d8949689-7df4-4ab2-b219-e6bc3ee79e40 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:35.448080979 +0000 UTC m=+6.577417190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret") pod "global-pull-secret-syncer-5f5cp" (UID: "d8949689-7df4-4ab2-b219-e6bc3ee79e40") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:35.048927 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:35.048776 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:35.049128 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:35.048962 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:35.049183 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:35.049159 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:35.049183 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:35.049178 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:35.049297 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:35.049191 2578 projected.go:194] Error preparing data for projected volume kube-api-access-w9tqr for pod openshift-network-diagnostics/network-check-target-94lpx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:35.049297 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:35.049252 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr podName:3a94647c-a5b5-4c9d-a13d-d6c824a3a79e nodeName:}" failed. No retries permitted until 2026-06-02 15:46:39.049232645 +0000 UTC m=+10.178568854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9tqr" (UniqueName: "kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr") pod "network-check-target-94lpx" (UID: "3a94647c-a5b5-4c9d-a13d-d6c824a3a79e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:35.049463 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:35.049324 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:35.049463 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:35.049359 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs podName:877796b1-2b26-4c87-903a-1235f441e838 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:39.049346672 +0000 UTC m=+10.178682887 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs") pod "network-metrics-daemon-7wvwq" (UID: "877796b1-2b26-4c87-903a-1235f441e838") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:35.451754 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:35.451664 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:35.451922 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:35.451810 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:35.451922 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:35.451890 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret podName:d8949689-7df4-4ab2-b219-e6bc3ee79e40 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:36.451867814 +0000 UTC m=+7.581204021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret") pod "global-pull-secret-syncer-5f5cp" (UID: "d8949689-7df4-4ab2-b219-e6bc3ee79e40") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:36.410359 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:36.410324 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:36.410827 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:36.410360 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:36.410827 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:36.410488 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:36.410827 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:36.410558 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:36.410827 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:36.410614 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:36.410827 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:36.410682 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:36.461009 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:36.460489 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:36.461009 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:36.460627 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:36.461009 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:36.460683 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret podName:d8949689-7df4-4ab2-b219-e6bc3ee79e40 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:38.46066516 +0000 UTC m=+9.590001364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret") pod "global-pull-secret-syncer-5f5cp" (UID: "d8949689-7df4-4ab2-b219-e6bc3ee79e40") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:38.410854 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:38.410820 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:38.410854 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:38.410852 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:38.411384 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:38.410828 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:38.411384 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:38.410984 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:38.411384 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:38.411092 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:38.411384 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:38.411189 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:38.478208 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:38.478177 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:38.478420 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:38.478355 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:38.478475 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:38.478450 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret podName:d8949689-7df4-4ab2-b219-e6bc3ee79e40 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:42.478429375 +0000 UTC m=+13.607765578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret") pod "global-pull-secret-syncer-5f5cp" (UID: "d8949689-7df4-4ab2-b219-e6bc3ee79e40") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:39.081824 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:39.081783 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:39.082009 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:39.081841 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:39.082009 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:39.081945 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:39.082009 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:39.081956 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:39.082009 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:39.081970 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:39.082009 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:39.081982 2578 projected.go:194] Error preparing data for projected volume kube-api-access-w9tqr for pod openshift-network-diagnostics/network-check-target-94lpx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:39.082009 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:39.082009 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs podName:877796b1-2b26-4c87-903a-1235f441e838 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:47.081990762 +0000 UTC m=+18.211326983 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs") pod "network-metrics-daemon-7wvwq" (UID: "877796b1-2b26-4c87-903a-1235f441e838") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:39.082269 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:39.082029 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr podName:3a94647c-a5b5-4c9d-a13d-d6c824a3a79e nodeName:}" failed. No retries permitted until 2026-06-02 15:46:47.08201852 +0000 UTC m=+18.211354723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9tqr" (UniqueName: "kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr") pod "network-check-target-94lpx" (UID: "3a94647c-a5b5-4c9d-a13d-d6c824a3a79e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:40.411645 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:40.411145 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:40.411645 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:40.411192 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:40.411645 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:40.411285 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:40.411645 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:40.411346 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:40.411645 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:40.411411 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:40.411645 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:40.411501 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:42.411668 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.411278 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:42.411668 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.411528 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:42.412584 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:42.411674 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:42.412584 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.411696 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:42.412584 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:42.411784 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:42.412584 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:42.411857 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:42.471724 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.471683 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-thdlh" event={"ID":"893b281a-ffc7-4fdd-b162-738f9f32da0c","Type":"ContainerStarted","Data":"0d5c289eaba2f5fbc1f963198ff7750051eb07a3b3ed237486f890610cd440c9"} Jun 02 15:46:42.473150 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.473120 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q4c6g" event={"ID":"2de345a5-85e0-422e-8497-e9d7c2fbdcc4","Type":"ContainerStarted","Data":"ac7f1dfcc35496c890bef5b562501de62ae5a125d800ee4a1d5b48857a234603"} Jun 02 15:46:42.474562 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.474534 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5ptj6" event={"ID":"d794b47c-d098-448e-926e-1e81d554e31f","Type":"ContainerStarted","Data":"ca97b5b69334180474c94fb87202f568955f25e1f1825440e05c0f604cb353dc"} Jun 02 15:46:42.475992 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.475956 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" event={"ID":"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5","Type":"ContainerStarted","Data":"c5210b948473c5375c92a87f2ade832f9ade613bb75070ac781eb296d83932e5"} Jun 02 15:46:42.477319 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.477297 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lvvg8" event={"ID":"6f2fd357-da07-4b2a-a98b-29ea324b4971","Type":"ContainerStarted","Data":"0990b3ebd2a463b3d5ef06ba4765d2883a3a6ba9946e787bea8dbd0d6ac4fadd"} Jun 02 15:46:42.478477 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.478453 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerStarted","Data":"73e396d63c7bccbc03e274bccc1daef07e27a8d487cc1548fcec4b1b2f020dfe"} Jun 02 15:46:42.488065 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.488016 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-thdlh" podStartSLOduration=3.66137563 podStartE2EDuration="13.488001425s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.069661854 +0000 UTC m=+3.198998055" lastFinishedPulling="2026-06-02 15:46:41.896287643 +0000 UTC m=+13.025623850" observedRunningTime="2026-06-02 15:46:42.487288124 +0000 UTC m=+13.616624357" watchObservedRunningTime="2026-06-02 15:46:42.488001425 +0000 UTC m=+13.617337649" Jun 02 15:46:42.512073 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.512038 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:42.512227 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:42.512194 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:42.512298 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:42.512262 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret podName:d8949689-7df4-4ab2-b219-e6bc3ee79e40 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:50.51224401 +0000 UTC m=+21.641580224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret") pod "global-pull-secret-syncer-5f5cp" (UID: "d8949689-7df4-4ab2-b219-e6bc3ee79e40") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:42.515028 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.514973 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-q4c6g" podStartSLOduration=3.713235271 podStartE2EDuration="13.51496279s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.068111501 +0000 UTC m=+3.197447716" lastFinishedPulling="2026-06-02 15:46:41.869839019 +0000 UTC m=+12.999175235" observedRunningTime="2026-06-02 15:46:42.500003799 +0000 UTC m=+13.629340025" watchObservedRunningTime="2026-06-02 15:46:42.51496279 +0000 UTC m=+13.644299013" Jun 02 15:46:42.515163 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.515062 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5ptj6" podStartSLOduration=3.711379954 podStartE2EDuration="13.515044958s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.065397359 +0000 UTC m=+3.194733561" lastFinishedPulling="2026-06-02 15:46:41.869062357 +0000 UTC m=+12.998398565" observedRunningTime="2026-06-02 15:46:42.51450638 +0000 UTC m=+13.643842603" watchObservedRunningTime="2026-06-02 15:46:42.515044958 +0000 UTC m=+13.644381182" Jun 02 15:46:42.560083 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:42.560020 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lvvg8" podStartSLOduration=3.795895087 podStartE2EDuration="13.560005258s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.079502877 +0000 UTC m=+3.208839088" lastFinishedPulling="2026-06-02 15:46:41.843613052 +0000 UTC m=+12.972949259" observedRunningTime="2026-06-02 15:46:42.532321499 +0000 UTC m=+13.661657722" watchObservedRunningTime="2026-06-02 15:46:42.560005258 +0000 UTC m=+13.689341482" Jun 02 15:46:43.482868 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:43.482821 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wpk7x" event={"ID":"010e3146-6f24-4c4f-853e-899ae1a2fe39","Type":"ContainerStarted","Data":"3189c69e67d56576619d823a4b08e92892f64252cae5483ee74b7af634a14e3d"} Jun 02 15:46:43.497185 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:43.497133 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wpk7x" podStartSLOduration=4.697798574 podStartE2EDuration="14.497118231s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.067697695 +0000 UTC m=+3.197033913" lastFinishedPulling="2026-06-02 15:46:41.867017354 +0000 UTC m=+12.996353570" observedRunningTime="2026-06-02 15:46:43.496631266 +0000 UTC m=+14.625967490" watchObservedRunningTime="2026-06-02 15:46:43.497118231 +0000 UTC m=+14.626454454" Jun 02 15:46:44.264053 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:44.264012 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:44.264967 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:44.264943 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:44.410789 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:44.410756 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:44.410989 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:44.410756 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:44.410989 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:44.410883 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:44.410989 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:44.410756 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:44.410989 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:44.410973 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:44.411178 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:44.411059 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:45.485480 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:45.485192 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 02 15:46:46.410585 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:46.410544 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:46.410585 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:46.410565 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:46.410863 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:46.410682 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:46.410863 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:46.410681 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:46.410863 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:46.410809 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:46.411013 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:46.410903 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:46.488241 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:46.488205 2578 generic.go:358] "Generic (PLEG): container finished" podID="e430bfcf-7490-4927-b6cc-f89f9d081f39" containerID="73e396d63c7bccbc03e274bccc1daef07e27a8d487cc1548fcec4b1b2f020dfe" exitCode=0 Jun 02 15:46:46.488695 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:46.488260 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerDied","Data":"73e396d63c7bccbc03e274bccc1daef07e27a8d487cc1548fcec4b1b2f020dfe"} Jun 02 15:46:47.150571 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:47.150522 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:47.150747 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:47.150689 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:47.150747 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:47.150697 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:47.150862 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:47.150797 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs podName:877796b1-2b26-4c87-903a-1235f441e838 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:03.150772844 +0000 UTC m=+34.280109069 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs") pod "network-metrics-daemon-7wvwq" (UID: "877796b1-2b26-4c87-903a-1235f441e838") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:47.150862 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:47.150852 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:47.150971 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:47.150876 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:47.150971 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:47.150890 2578 projected.go:194] Error preparing data for projected volume kube-api-access-w9tqr for pod openshift-network-diagnostics/network-check-target-94lpx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:47.150971 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:47.150951 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr podName:3a94647c-a5b5-4c9d-a13d-d6c824a3a79e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:03.150934755 +0000 UTC m=+34.280270963 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9tqr" (UniqueName: "kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr") pod "network-check-target-94lpx" (UID: "3a94647c-a5b5-4c9d-a13d-d6c824a3a79e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:48.411452 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:48.411390 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:48.411966 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:48.411391 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:48.411966 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:48.411512 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:48.411966 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:48.411394 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:48.411966 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:48.411597 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:48.411966 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:48.411690 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:50.411423 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.411092 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:50.411423 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:50.411227 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:50.411423 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.411357 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:50.412068 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:50.411458 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:50.412068 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.411567 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:50.412068 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:50.411655 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:50.415698 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.415547 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 15:46:50.497732 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.497696 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" event={"ID":"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5","Type":"ContainerStarted","Data":"8a9752f9181f11aebb6eb79be32f0c200976ef1051f75a016e009d846dcea3c0"} Jun 02 15:46:50.499905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.499860 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"c5443d3244d85c4f268d83ffae186125da8d938a2aafbab044ffbcc80ba2104f"} Jun 02 15:46:50.499905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.499901 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"7bfe73758cb5acb29498e83d9d27e44dbf950220357019afef2d3d1f3772c6c8"} Jun 02 15:46:50.501659 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.501634 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zc6zt" event={"ID":"7a43ecc6-b5eb-435b-9737-ee046a1013ab","Type":"ContainerStarted","Data":"33e8497983eb2283010558b6e9e6a229e6a52b256dc7a80e3bd2e94e5cc9805a"} Jun 02 15:46:50.517158 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.517113 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-zc6zt" podStartSLOduration=3.32848157 podStartE2EDuration="21.51709933s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.073137486 +0000 UTC m=+3.202473703" lastFinishedPulling="2026-06-02 15:46:50.261755248 +0000 UTC m=+21.391091463" observedRunningTime="2026-06-02 15:46:50.516358981 +0000 UTC m=+21.645695204" watchObservedRunningTime="2026-06-02 15:46:50.51709933 +0000 UTC m=+21.646435552" Jun 02 15:46:50.570873 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:50.570845 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:50.571019 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:50.570997 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:50.571096 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:50.571068 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret podName:d8949689-7df4-4ab2-b219-e6bc3ee79e40 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:06.571048401 +0000 UTC m=+37.700384602 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret") pod "global-pull-secret-syncer-5f5cp" (UID: "d8949689-7df4-4ab2-b219-e6bc3ee79e40") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:51.378231 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:51.378148 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T15:46:50.415564823Z","UUID":"eb58e7c0-7e71-4d0b-9ecc-9cf8fe6175b1","Handler":null,"Name":"","Endpoint":""} Jun 02 15:46:51.380337 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:51.380314 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 15:46:51.380462 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:51.380348 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 15:46:51.504771 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:51.504730 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" event={"ID":"9c7ecca4-9300-4a1c-8f78-b999fe91d8e5","Type":"ContainerStarted","Data":"240186cf019c59cede6b147eaaba6b4ca270a347bec906160e1b81e469cc9f3d"} Jun 02 15:46:51.507176 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:51.507153 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"c603f497a8afa67f1fb4862fd07d1eeac876e03cd356f1f4d51c4d935e18e99c"} Jun 02 15:46:51.507176 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:51.507178 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"4618a33df79c5ac1bb1ca33cb7e9081850b825a8354b615743b86c14805eabf1"} Jun 02 15:46:51.507398 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:51.507187 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"8a292fefb82d47cc4ee61b3761d352b41ab10166c70b0f2bffb5d22c58806812"} Jun 02 15:46:51.507398 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:51.507196 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"d10d3d63a6e36445b4f64cf563ba405ba88fae31e3f3c32dfda52e460a323136"} Jun 02 15:46:52.410471 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:52.410442 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:52.410651 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:52.410557 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:52.410651 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:52.410562 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:52.410764 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:52.410684 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:52.410764 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:52.410730 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:52.410850 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:52.410825 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:53.513527 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:53.513487 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"d49944f48bb5478101ffa3c0d7a89e9b0adc86dce0ed05030db38500b246b07f"} Jun 02 15:46:54.410560 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:54.410523 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:54.410560 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:54.410544 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:54.410788 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:54.410529 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:54.410788 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:54.410634 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:54.410788 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:54.410718 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:54.410899 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:54.410781 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:56.410647 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.410615 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:56.411106 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.410652 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:56.411106 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.410651 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:56.411106 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:56.410718 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:56.411106 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:56.410810 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:56.411106 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:56.410888 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:56.522422 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.522385 2578 generic.go:358] "Generic (PLEG): container finished" podID="e430bfcf-7490-4927-b6cc-f89f9d081f39" containerID="b408f725d0847e32eff02ef0017e26d152bde953546fee714406dc2f664dca99" exitCode=0 Jun 02 15:46:56.522563 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.522457 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerDied","Data":"b408f725d0847e32eff02ef0017e26d152bde953546fee714406dc2f664dca99"} Jun 02 15:46:56.525699 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.525675 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" event={"ID":"6d0b6596-6b7d-4228-85d3-95620e2804a7","Type":"ContainerStarted","Data":"49c5a73f58d644713e6b7a219e56fd1f47d4acfdc1c6959703e7e7962a1cd68e"} Jun 02 15:46:56.525977 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.525958 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:56.526077 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.525989 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:56.541009 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.540985 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:56.554477 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:56.554438 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mgdbt" podStartSLOduration=8.270302813 podStartE2EDuration="27.554426901s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.078316329 +0000 UTC m=+3.207652534" lastFinishedPulling="2026-06-02 15:46:51.362440406 +0000 UTC m=+22.491776622" observedRunningTime="2026-06-02 15:46:51.529506918 +0000 UTC m=+22.658843141" watchObservedRunningTime="2026-06-02 15:46:56.554426901 +0000 UTC m=+27.683763124" Jun 02 15:46:57.529135 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.529108 2578 generic.go:358] "Generic (PLEG): container finished" podID="e430bfcf-7490-4927-b6cc-f89f9d081f39" containerID="fbc71e72e4e1be035dda2041b04dfe2823d0b862507d9fa27480fd690db75f8a" exitCode=0 Jun 02 15:46:57.529471 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.529177 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerDied","Data":"fbc71e72e4e1be035dda2041b04dfe2823d0b862507d9fa27480fd690db75f8a"} Jun 02 15:46:57.529808 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.529790 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:57.544889 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.544868 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:46:57.553696 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.553651 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" podStartSLOduration=10.366979682 podStartE2EDuration="28.553637776s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.07549606 +0000 UTC m=+3.204832267" lastFinishedPulling="2026-06-02 15:46:50.26215416 +0000 UTC m=+21.391490361" observedRunningTime="2026-06-02 15:46:56.585480388 +0000 UTC m=+27.714816611" watchObservedRunningTime="2026-06-02 15:46:57.553637776 +0000 UTC m=+28.682973998" Jun 02 15:46:57.849503 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.849472 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7wvwq"] Jun 02 15:46:57.849651 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.849600 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:57.849701 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:57.849682 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:57.855671 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.854172 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5f5cp"] Jun 02 15:46:57.855671 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.854325 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:57.855671 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:57.854472 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:57.856949 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.856913 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-94lpx"] Jun 02 15:46:57.857066 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:57.857036 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:57.857160 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:57.857125 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:59.411837 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:59.411606 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:46:59.412255 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:59.411667 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:46:59.412255 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:59.411923 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:46:59.412255 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:59.411688 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:46:59.412255 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:59.411984 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:46:59.412255 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:46:59.412071 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:46:59.535823 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:59.535792 2578 generic.go:358] "Generic (PLEG): container finished" podID="e430bfcf-7490-4927-b6cc-f89f9d081f39" containerID="7a7cf719fa6539fc049319ef145241e2f262a4fa5bd188915ded0410194167d0" exitCode=0 Jun 02 15:46:59.535992 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:59.535871 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerDied","Data":"7a7cf719fa6539fc049319ef145241e2f262a4fa5bd188915ded0410194167d0"} Jun 02 15:46:59.605453 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:59.605404 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:46:59.605657 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:59.605570 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 02 15:46:59.605991 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:46:59.605973 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lvvg8" Jun 02 15:47:01.410405 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:01.410358 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:47:01.410900 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:01.410418 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:47:01.410900 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:01.410358 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:47:01.410900 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:01.410488 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-94lpx" podUID="3a94647c-a5b5-4c9d-a13d-d6c824a3a79e" Jun 02 15:47:01.410900 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:01.410574 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5f5cp" podUID="d8949689-7df4-4ab2-b219-e6bc3ee79e40" Jun 02 15:47:01.410900 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:01.410684 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7wvwq" podUID="877796b1-2b26-4c87-903a-1235f441e838" Jun 02 15:47:03.167809 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.167743 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:47:03.168357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.167818 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:47:03.168357 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.167916 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:47:03.168357 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.167999 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs podName:877796b1-2b26-4c87-903a-1235f441e838 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.167975803 +0000 UTC m=+66.297312008 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs") pod "network-metrics-daemon-7wvwq" (UID: "877796b1-2b26-4c87-903a-1235f441e838") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:47:03.168357 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.168007 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:47:03.168357 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.168031 2578 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:47:03.168357 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.168044 2578 projected.go:194] Error preparing data for projected volume kube-api-access-w9tqr for pod openshift-network-diagnostics/network-check-target-94lpx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:47:03.168357 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.168101 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr podName:3a94647c-a5b5-4c9d-a13d-d6c824a3a79e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.16808592 +0000 UTC m=+66.297422136 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w9tqr" (UniqueName: "kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr") pod "network-check-target-94lpx" (UID: "3a94647c-a5b5-4c9d-a13d-d6c824a3a79e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:47:03.173627 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.173594 2578 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-123.ec2.internal" event="NodeReady" Jun 02 15:47:03.173777 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.173744 2578 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 15:47:03.226729 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.226692 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84b49587fd-824gx"] Jun 02 15:47:03.235927 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.235890 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25"] Jun 02 15:47:03.236986 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.236958 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.239419 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.239391 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 15:47:03.239848 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.239827 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 15:47:03.240141 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.240124 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 15:47:03.240623 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.240593 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vhj6z\"" Jun 02 15:47:03.245202 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.244462 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6985979865-kdkr9"] Jun 02 15:47:03.245202 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.244721 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:03.247737 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.247714 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.248208 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.248184 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 02 15:47:03.248322 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.248303 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 15:47:03.249353 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.249330 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wp9pm\"" Jun 02 15:47:03.249591 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.249574 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.252112 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.252082 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h"] Jun 02 15:47:03.252228 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.252152 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.257527 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.257505 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-x2kvd\"" Jun 02 15:47:03.257851 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.257833 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 02 15:47:03.258396 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.258377 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.258648 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.258631 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 02 15:47:03.258953 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.258936 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 02 15:47:03.259257 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.259241 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 02 15:47:03.260431 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.260408 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.261797 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.261777 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp"] Jun 02 15:47:03.262259 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.261941 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.266486 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.266437 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 02 15:47:03.266587 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.266494 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 02 15:47:03.266587 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.266540 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-tw5hb\"" Jun 02 15:47:03.266705 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.266615 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.266796 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.266775 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.270117 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.269871 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb"] Jun 02 15:47:03.270117 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.270069 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.272463 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.272443 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.272576 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.272528 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6qxsg\"" Jun 02 15:47:03.272576 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.272550 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 02 15:47:03.272872 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.272770 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 02 15:47:03.272987 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.272882 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.278830 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.277091 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn"] Jun 02 15:47:03.278830 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.277637 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb" Jun 02 15:47:03.279737 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.279719 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.285244 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.285221 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bc2vp\"" Jun 02 15:47:03.285973 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.285955 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.287786 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.287763 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-788bf58df5-chj4l"] Jun 02 15:47:03.294954 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.294931 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h"] Jun 02 15:47:03.295061 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.294961 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84b49587fd-824gx"] Jun 02 15:47:03.295061 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.294972 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25"] Jun 02 15:47:03.295061 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.294980 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb"] Jun 02 15:47:03.295061 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.294993 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb"] Jun 02 15:47:03.295061 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.295030 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.295312 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.295117 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.301357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.301328 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vxmkc\"" Jun 02 15:47:03.301904 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.301548 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 02 15:47:03.301904 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.301819 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.301904 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.301830 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.302470 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.302420 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 02 15:47:03.304176 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.302636 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kvplc\"" Jun 02 15:47:03.304176 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.302966 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 02 15:47:03.304176 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.303391 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.304986 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.304968 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 02 15:47:03.305985 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.305945 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-khtbj"] Jun 02 15:47:03.306685 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.306620 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.311484 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.311427 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6985979865-kdkr9"] Jun 02 15:47:03.311484 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.311454 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r8wp6"] Jun 02 15:47:03.311649 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.311521 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb" Jun 02 15:47:03.311649 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.311572 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.313510 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.313492 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 02 15:47:03.314661 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.314610 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 02 15:47:03.315029 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.315009 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f4vs4\"" Jun 02 15:47:03.315126 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.315056 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.315890 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.315470 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-p4flm\"" Jun 02 15:47:03.315890 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.315499 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.315890 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.315713 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 02 15:47:03.315890 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.315729 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.316466 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.316445 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.317775 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.317755 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5jxnm"] Jun 02 15:47:03.318286 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.318007 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:03.320516 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.320486 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 15:47:03.320912 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.320896 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 15:47:03.321163 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.321144 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 15:47:03.321247 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.320898 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ws8nx\"" Jun 02 15:47:03.323060 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.323041 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 02 15:47:03.324269 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.324250 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp"] Jun 02 15:47:03.324466 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.324440 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.324551 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.324536 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r8wp6"] Jun 02 15:47:03.324675 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.324628 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5jxnm"] Jun 02 15:47:03.325232 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.325203 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-khtbj"] Jun 02 15:47:03.325350 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.325238 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-chj4l"] Jun 02 15:47:03.325350 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.325250 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn"] Jun 02 15:47:03.325350 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.325265 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb"] Jun 02 15:47:03.326203 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.326183 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g6cp8\"" Jun 02 15:47:03.326289 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.326209 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 15:47:03.326613 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.326596 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 15:47:03.369206 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-installation-pull-secrets\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.369206 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369205 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87b11978-f71a-49d3-bf1e-2eb41d30143b-serving-cert\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.369437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369227 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-bound-sa-token\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.369437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369267 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/737dbd8f-f1e2-4152-90a0-c8b86b42a170-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.369437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9b9abd4-4ecd-44ca-9329-a1923368bd46-serving-cert\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.369437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369326 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9b9abd4-4ecd-44ca-9329-a1923368bd46-tmp\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.369437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b9abd4-4ecd-44ca-9329-a1923368bd46-service-ca-bundle\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.369437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369358 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.369437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369390 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-default-certificate\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.369437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqkmm\" (UniqueName: \"kubernetes.io/projected/2b93a739-0f4e-4492-9616-9f2ae397ecd0-kube-api-access-mqkmm\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87b11978-f71a-49d3-bf1e-2eb41d30143b-config\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbbcc\" (UniqueName: \"kubernetes.io/projected/737dbd8f-f1e2-4152-90a0-c8b86b42a170-kube-api-access-tbbcc\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8w26\" (UniqueName: \"kubernetes.io/projected/0cd57594-0419-401e-8406-38e65a8e0ef5-kube-api-access-f8w26\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369531 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b9abd4-4ecd-44ca-9329-a1923368bd46-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tln8h\" (UniqueName: \"kubernetes.io/projected/87b11978-f71a-49d3-bf1e-2eb41d30143b-kube-api-access-tln8h\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369565 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cd57594-0419-401e-8406-38e65a8e0ef5-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369585 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cd57594-0419-401e-8406-38e65a8e0ef5-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-certificates\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.369703 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/626b807d-b328-4e35-9d57-48fe8ecc46f4-ca-trust-extracted\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-image-registry-private-configuration\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-trusted-ca\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369761 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b9b9abd4-4ecd-44ca-9329-a1923368bd46-snapshots\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnj9s\" (UniqueName: \"kubernetes.io/projected/ed7580d5-42f8-4a02-aa56-1f5b15dff6b1-kube-api-access-gnj9s\") pod \"volume-data-source-validator-7598fd5bd-77fmb\" (UID: \"ed7580d5-42f8-4a02-aa56-1f5b15dff6b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369881 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksbc6\" (UniqueName: \"kubernetes.io/projected/b9b9abd4-4ecd-44ca-9329-a1923368bd46-kube-api-access-ksbc6\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-stats-auth\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdpjw\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-kube-api-access-cdpjw\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.370018 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.369975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr84v\" (UniqueName: \"kubernetes.io/projected/e1de098e-0945-4d7d-bb6e-9d97f8e47765-kube-api-access-zr84v\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:03.410417 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.410383 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:47:03.410417 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.410417 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:47:03.410644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.410389 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:47:03.412936 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.412898 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 15:47:03.412936 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.412901 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2vmzq\"" Jun 02 15:47:03.413143 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.412954 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6qg5h\"" Jun 02 15:47:03.413143 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.412903 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 15:47:03.471023 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.470981 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-stats-auth\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.471238 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471210 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cdpjw\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-kube-api-access-cdpjw\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.471338 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471252 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zr84v\" (UniqueName: \"kubernetes.io/projected/e1de098e-0945-4d7d-bb6e-9d97f8e47765-kube-api-access-zr84v\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:03.471338 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471300 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-installation-pull-secrets\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.471338 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd41e666-70e5-4b55-b2d2-bcb8e142678e-config-volume\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.471513 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471378 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87b11978-f71a-49d3-bf1e-2eb41d30143b-serving-cert\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.471513 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-config\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.471513 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471437 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-bound-sa-token\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.471513 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471463 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/737dbd8f-f1e2-4152-90a0-c8b86b42a170-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.471719 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59f45\" (UniqueName: \"kubernetes.io/projected/46349232-ad6b-4e6d-851e-53efe1621539-kube-api-access-59f45\") pod \"network-check-source-56d84b9f44-98rxb\" (UID: \"46349232-ad6b-4e6d-851e-53efe1621539\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb" Jun 02 15:47:03.471835 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471817 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9b9abd4-4ecd-44ca-9329-a1923368bd46-serving-cert\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.471978 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.471962 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9b9abd4-4ecd-44ca-9329-a1923368bd46-tmp\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.472401 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472355 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b9abd4-4ecd-44ca-9329-a1923368bd46-service-ca-bundle\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.472500 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472418 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.472500 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472447 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-default-certificate\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.472500 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472474 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mqkmm\" (UniqueName: \"kubernetes.io/projected/2b93a739-0f4e-4492-9616-9f2ae397ecd0-kube-api-access-mqkmm\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.472651 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-trusted-ca\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.472651 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472534 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87b11978-f71a-49d3-bf1e-2eb41d30143b-config\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.472651 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57qk2\" (UniqueName: \"kubernetes.io/projected/dd41e666-70e5-4b55-b2d2-bcb8e142678e-kube-api-access-57qk2\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.472651 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472599 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tbbcc\" (UniqueName: \"kubernetes.io/projected/737dbd8f-f1e2-4152-90a0-c8b86b42a170-kube-api-access-tbbcc\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.472651 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-serving-cert\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472650 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472681 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f8w26\" (UniqueName: \"kubernetes.io/projected/0cd57594-0419-401e-8406-38e65a8e0ef5-kube-api-access-f8w26\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472708 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b9abd4-4ecd-44ca-9329-a1923368bd46-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472743 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tln8h\" (UniqueName: \"kubernetes.io/projected/87b11978-f71a-49d3-bf1e-2eb41d30143b-kube-api-access-tln8h\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472771 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cd57594-0419-401e-8406-38e65a8e0ef5-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472797 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cd57594-0419-401e-8406-38e65a8e0ef5-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472824 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472850 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-certificates\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.472905 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8dm7\" (UniqueName: \"kubernetes.io/projected/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-kube-api-access-m8dm7\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd41e666-70e5-4b55-b2d2-bcb8e142678e-tmp-dir\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67kq4\" (UniqueName: \"kubernetes.io/projected/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-kube-api-access-67kq4\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472993 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/626b807d-b328-4e35-9d57-48fe8ecc46f4-ca-trust-extracted\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473028 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473083 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473130 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-image-registry-private-configuration\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473157 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-trusted-ca\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473186 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473229 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b9b9abd4-4ecd-44ca-9329-a1923368bd46-snapshots\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473257 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gnj9s\" (UniqueName: \"kubernetes.io/projected/ed7580d5-42f8-4a02-aa56-1f5b15dff6b1-kube-api-access-gnj9s\") pod \"volume-data-source-validator-7598fd5bd-77fmb\" (UID: \"ed7580d5-42f8-4a02-aa56-1f5b15dff6b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473288 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ksbc6\" (UniqueName: \"kubernetes.io/projected/b9b9abd4-4ecd-44ca-9329-a1923368bd46-kube-api-access-ksbc6\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.473327 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.473315 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.473975 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.473513 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 15:47:03.473975 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.473587 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls podName:e1de098e-0945-4d7d-bb6e-9d97f8e47765 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:03.973561268 +0000 UTC m=+35.102897485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-4kq25" (UID: "e1de098e-0945-4d7d-bb6e-9d97f8e47765") : secret "samples-operator-tls" not found Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.472560 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/737dbd8f-f1e2-4152-90a0-c8b86b42a170-telemetry-config\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.476300 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.476389 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:03.976353692 +0000 UTC m=+35.105689905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : secret "router-metrics-certs-default" not found Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.476405 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87b11978-f71a-49d3-bf1e-2eb41d30143b-config\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.476700 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.476716 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84b49587fd-824gx: secret "image-registry-tls" not found Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.476816 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls podName:626b807d-b328-4e35-9d57-48fe8ecc46f4 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:03.976795731 +0000 UTC m=+35.106131944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls") pod "image-registry-84b49587fd-824gx" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4") : secret "image-registry-tls" not found Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.476829 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-stats-auth\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.476896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b9b9abd4-4ecd-44ca-9329-a1923368bd46-snapshots\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.477005 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:03.976985873 +0000 UTC m=+35.106322097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : configmap references non-existent config key: service-ca.crt Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477011 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0cd57594-0419-401e-8406-38e65a8e0ef5-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.477124 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477136 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9b9abd4-4ecd-44ca-9329-a1923368bd46-tmp\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.477182 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls podName:737dbd8f-f1e2-4152-90a0-c8b86b42a170 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:03.977167751 +0000 UTC m=+35.106503965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-s9svn" (UID: "737dbd8f-f1e2-4152-90a0-c8b86b42a170") : secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:03.477624 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477422 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/626b807d-b328-4e35-9d57-48fe8ecc46f4-ca-trust-extracted\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.478389 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477571 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b9abd4-4ecd-44ca-9329-a1923368bd46-service-ca-bundle\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.478389 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477584 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-installation-pull-secrets\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.478389 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477630 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0cd57594-0419-401e-8406-38e65a8e0ef5-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.478389 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477669 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-certificates\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.478389 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477898 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9b9abd4-4ecd-44ca-9329-a1923368bd46-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.478389 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.477977 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-trusted-ca\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.478389 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.478141 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87b11978-f71a-49d3-bf1e-2eb41d30143b-serving-cert\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.478389 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.478262 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9b9abd4-4ecd-44ca-9329-a1923368bd46-serving-cert\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.478756 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.478617 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-default-certificate\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.479902 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.479877 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-image-registry-private-configuration\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.483191 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.483160 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr84v\" (UniqueName: \"kubernetes.io/projected/e1de098e-0945-4d7d-bb6e-9d97f8e47765-kube-api-access-zr84v\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:03.483810 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.483787 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdpjw\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-kube-api-access-cdpjw\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.484729 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.484697 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-bound-sa-token\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.485934 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.485887 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbbcc\" (UniqueName: \"kubernetes.io/projected/737dbd8f-f1e2-4152-90a0-c8b86b42a170-kube-api-access-tbbcc\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.488146 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.488091 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8w26\" (UniqueName: \"kubernetes.io/projected/0cd57594-0419-401e-8406-38e65a8e0ef5-kube-api-access-f8w26\") pod \"kube-storage-version-migrator-operator-96ddc787c-2q62h\" (UID: \"0cd57594-0419-401e-8406-38e65a8e0ef5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.488524 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.488501 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqkmm\" (UniqueName: \"kubernetes.io/projected/2b93a739-0f4e-4492-9616-9f2ae397ecd0-kube-api-access-mqkmm\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.489377 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.489340 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tln8h\" (UniqueName: \"kubernetes.io/projected/87b11978-f71a-49d3-bf1e-2eb41d30143b-kube-api-access-tln8h\") pod \"service-ca-operator-69877444c-7zqxp\" (UID: \"87b11978-f71a-49d3-bf1e-2eb41d30143b\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.489481 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.489421 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnj9s\" (UniqueName: \"kubernetes.io/projected/ed7580d5-42f8-4a02-aa56-1f5b15dff6b1-kube-api-access-gnj9s\") pod \"volume-data-source-validator-7598fd5bd-77fmb\" (UID: \"ed7580d5-42f8-4a02-aa56-1f5b15dff6b1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb" Jun 02 15:47:03.490769 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.490741 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksbc6\" (UniqueName: \"kubernetes.io/projected/b9b9abd4-4ecd-44ca-9329-a1923368bd46-kube-api-access-ksbc6\") pod \"insights-operator-788bf58df5-chj4l\" (UID: \"b9b9abd4-4ecd-44ca-9329-a1923368bd46\") " pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.574151 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574069 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-serving-cert\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.574151 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574112 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.574357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574161 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m8dm7\" (UniqueName: \"kubernetes.io/projected/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-kube-api-access-m8dm7\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.574357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574225 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd41e666-70e5-4b55-b2d2-bcb8e142678e-tmp-dir\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.574357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574267 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-67kq4\" (UniqueName: \"kubernetes.io/projected/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-kube-api-access-67kq4\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:03.574357 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.574301 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:47:03.574357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574314 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:03.574622 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.574393 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls podName:dd41e666-70e5-4b55-b2d2-bcb8e142678e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.074353289 +0000 UTC m=+35.203689496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls") pod "dns-default-5jxnm" (UID: "dd41e666-70e5-4b55-b2d2-bcb8e142678e") : secret "dns-default-metrics-tls" not found Jun 02 15:47:03.574622 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.574419 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:47:03.574622 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.574463 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert podName:6dd709d8-d6a3-42aa-bef6-9fa70025b49b nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.074448459 +0000 UTC m=+35.203784661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert") pod "ingress-canary-r8wp6" (UID: "6dd709d8-d6a3-42aa-bef6-9fa70025b49b") : secret "canary-serving-cert" not found Jun 02 15:47:03.574622 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574512 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd41e666-70e5-4b55-b2d2-bcb8e142678e-config-volume\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.574622 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574539 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-config\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.574622 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574570 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-59f45\" (UniqueName: \"kubernetes.io/projected/46349232-ad6b-4e6d-851e-53efe1621539-kube-api-access-59f45\") pod \"network-check-source-56d84b9f44-98rxb\" (UID: \"46349232-ad6b-4e6d-851e-53efe1621539\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb" Jun 02 15:47:03.574916 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574623 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-trusted-ca\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.574916 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574661 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-57qk2\" (UniqueName: \"kubernetes.io/projected/dd41e666-70e5-4b55-b2d2-bcb8e142678e-kube-api-access-57qk2\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.575021 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.574914 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd41e666-70e5-4b55-b2d2-bcb8e142678e-tmp-dir\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.575435 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.575409 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-config\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.575435 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.575425 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd41e666-70e5-4b55-b2d2-bcb8e142678e-config-volume\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.575707 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.575682 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-trusted-ca\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.576860 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.576840 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-serving-cert\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.579779 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.579762 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" Jun 02 15:47:03.585562 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.585508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59f45\" (UniqueName: \"kubernetes.io/projected/46349232-ad6b-4e6d-851e-53efe1621539-kube-api-access-59f45\") pod \"network-check-source-56d84b9f44-98rxb\" (UID: \"46349232-ad6b-4e6d-851e-53efe1621539\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb" Jun 02 15:47:03.585562 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.585529 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67kq4\" (UniqueName: \"kubernetes.io/projected/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-kube-api-access-67kq4\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:03.586179 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.586155 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57qk2\" (UniqueName: \"kubernetes.io/projected/dd41e666-70e5-4b55-b2d2-bcb8e142678e-kube-api-access-57qk2\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:03.586653 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.586636 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8dm7\" (UniqueName: \"kubernetes.io/projected/e3ff6dbe-9cfd-4e96-9b58-cc8c07878974-kube-api-access-m8dm7\") pod \"console-operator-86d7c6cc77-khtbj\" (UID: \"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974\") " pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.590445 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.590390 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" Jun 02 15:47:03.600224 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.600189 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb" Jun 02 15:47:03.620133 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.620101 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn"] Jun 02 15:47:03.622879 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.622849 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-chj4l" Jun 02 15:47:03.629573 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.629544 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:03.637382 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.637328 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb" Jun 02 15:47:03.645837 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.645808 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn"] Jun 02 15:47:03.645993 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.645943 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:03.648610 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.648587 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 02 15:47:03.648775 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.648640 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-slmnc\"" Jun 02 15:47:03.648932 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.648914 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 02 15:47:03.777144 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.777105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efe6293b-b3ab-49e5-ada2-61884c0bed13-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:03.777347 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.777323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:03.878709 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.878613 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:03.878885 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.878740 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efe6293b-b3ab-49e5-ada2-61884c0bed13-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:03.878885 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.878794 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:47:03.878885 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.878884 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert podName:efe6293b-b3ab-49e5-ada2-61884c0bed13 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.378858178 +0000 UTC m=+35.508194384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-hp7hn" (UID: "efe6293b-b3ab-49e5-ada2-61884c0bed13") : secret "networking-console-plugin-cert" not found Jun 02 15:47:03.879639 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.879607 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/efe6293b-b3ab-49e5-ada2-61884c0bed13-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:03.980288 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.980086 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:03.980503 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980242 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:03.980503 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.980380 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:03.980503 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980420 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls podName:737dbd8f-f1e2-4152-90a0-c8b86b42a170 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.980399508 +0000 UTC m=+36.109735709 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-s9svn" (UID: "737dbd8f-f1e2-4152-90a0-c8b86b42a170") : secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:03.980503 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980478 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 15:47:03.980503 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.980482 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:03.980503 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980492 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84b49587fd-824gx: secret "image-registry-tls" not found Jun 02 15:47:03.980797 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.980526 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.980797 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980549 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls podName:626b807d-b328-4e35-9d57-48fe8ecc46f4 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.980532573 +0000 UTC m=+36.109868773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls") pod "image-registry-84b49587fd-824gx" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4") : secret "image-registry-tls" not found Jun 02 15:47:03.980797 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980597 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 15:47:03.980797 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980607 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 15:47:03.980797 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:03.980600 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:03.980797 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980645 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.980632544 +0000 UTC m=+36.109968745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : secret "router-metrics-certs-default" not found Jun 02 15:47:03.980797 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980669 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls podName:e1de098e-0945-4d7d-bb6e-9d97f8e47765 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.980660641 +0000 UTC m=+36.109996849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-4kq25" (UID: "e1de098e-0945-4d7d-bb6e-9d97f8e47765") : secret "samples-operator-tls" not found Jun 02 15:47:03.980797 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:03.980688 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:04.980678671 +0000 UTC m=+36.110014893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : configmap references non-existent config key: service-ca.crt Jun 02 15:47:04.081526 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:04.081470 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:04.081720 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.081649 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:47:04.081720 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:04.081668 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:04.081838 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.081726 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls podName:dd41e666-70e5-4b55-b2d2-bcb8e142678e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:05.081704185 +0000 UTC m=+36.211040406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls") pod "dns-default-5jxnm" (UID: "dd41e666-70e5-4b55-b2d2-bcb8e142678e") : secret "dns-default-metrics-tls" not found Jun 02 15:47:04.081838 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.081778 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:47:04.081838 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.081826 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert podName:6dd709d8-d6a3-42aa-bef6-9fa70025b49b nodeName:}" failed. No retries permitted until 2026-06-02 15:47:05.081810473 +0000 UTC m=+36.211146674 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert") pod "ingress-canary-r8wp6" (UID: "6dd709d8-d6a3-42aa-bef6-9fa70025b49b") : secret "canary-serving-cert" not found Jun 02 15:47:04.385353 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:04.385311 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:04.385916 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.385505 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:47:04.385916 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.385586 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert podName:efe6293b-b3ab-49e5-ada2-61884c0bed13 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:05.385569127 +0000 UTC m=+36.514905332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-hp7hn" (UID: "efe6293b-b3ab-49e5-ada2-61884c0bed13") : secret "networking-console-plugin-cert" not found Jun 02 15:47:04.990061 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:04.990014 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:04.990257 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:04.990086 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:04.990257 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:04.990122 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:04.990257 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:04.990156 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:04.990257 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990176 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 15:47:04.990257 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990201 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84b49587fd-824gx: secret "image-registry-tls" not found Jun 02 15:47:04.990257 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990235 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 15:47:04.990257 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:04.990240 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:04.990257 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990254 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 15:47:04.990708 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990266 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls podName:626b807d-b328-4e35-9d57-48fe8ecc46f4 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:06.990245415 +0000 UTC m=+38.119581639 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls") pod "image-registry-84b49587fd-824gx" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4") : secret "image-registry-tls" not found Jun 02 15:47:04.990708 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990315 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:04.990708 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990337 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:06.990319352 +0000 UTC m=+38.119655569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : secret "router-metrics-certs-default" not found Jun 02 15:47:04.990708 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990377 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:06.990353136 +0000 UTC m=+38.119689337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : configmap references non-existent config key: service-ca.crt Jun 02 15:47:04.990708 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990430 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls podName:e1de098e-0945-4d7d-bb6e-9d97f8e47765 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:06.990390595 +0000 UTC m=+38.119726804 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-4kq25" (UID: "e1de098e-0945-4d7d-bb6e-9d97f8e47765") : secret "samples-operator-tls" not found Jun 02 15:47:04.990708 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:04.990453 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls podName:737dbd8f-f1e2-4152-90a0-c8b86b42a170 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:06.990442774 +0000 UTC m=+38.119778977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-s9svn" (UID: "737dbd8f-f1e2-4152-90a0-c8b86b42a170") : secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:05.091987 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.091949 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:05.092155 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.092042 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:05.092155 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:05.092119 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:47:05.092267 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:05.092202 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls podName:dd41e666-70e5-4b55-b2d2-bcb8e142678e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:07.092178656 +0000 UTC m=+38.221514897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls") pod "dns-default-5jxnm" (UID: "dd41e666-70e5-4b55-b2d2-bcb8e142678e") : secret "dns-default-metrics-tls" not found Jun 02 15:47:05.092267 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:05.092200 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:47:05.092267 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:05.092264 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert podName:6dd709d8-d6a3-42aa-bef6-9fa70025b49b nodeName:}" failed. No retries permitted until 2026-06-02 15:47:07.092252879 +0000 UTC m=+38.221589094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert") pod "ingress-canary-r8wp6" (UID: "6dd709d8-d6a3-42aa-bef6-9fa70025b49b") : secret "canary-serving-cert" not found Jun 02 15:47:05.394619 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.394572 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:05.395017 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:05.394713 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:47:05.395017 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:05.394781 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert podName:efe6293b-b3ab-49e5-ada2-61884c0bed13 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:07.394765582 +0000 UTC m=+38.524101786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-hp7hn" (UID: "efe6293b-b3ab-49e5-ada2-61884c0bed13") : secret "networking-console-plugin-cert" not found Jun 02 15:47:05.778075 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.778014 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb"] Jun 02 15:47:05.779237 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.779190 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-chj4l"] Jun 02 15:47:05.788326 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.788302 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp"] Jun 02 15:47:05.791217 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.791195 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb"] Jun 02 15:47:05.795063 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.795026 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-khtbj"] Jun 02 15:47:05.800058 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:05.800033 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h"] Jun 02 15:47:05.814453 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:05.814428 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b9abd4_4ecd_44ca_9329_a1923368bd46.slice/crio-883b0a179bad7e47501afad9b0781f922f4d521dac03e8983b8631cc8a8ec0f2 WatchSource:0}: Error finding container 883b0a179bad7e47501afad9b0781f922f4d521dac03e8983b8631cc8a8ec0f2: Status 404 returned error can't find the container with id 883b0a179bad7e47501afad9b0781f922f4d521dac03e8983b8631cc8a8ec0f2 Jun 02 15:47:05.822958 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:05.822931 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46349232_ad6b_4e6d_851e_53efe1621539.slice/crio-4536f2f237740c400d44f3274fe7bb5bca20ac0fc6649cb8d964186ed818e2fc WatchSource:0}: Error finding container 4536f2f237740c400d44f3274fe7bb5bca20ac0fc6649cb8d964186ed818e2fc: Status 404 returned error can't find the container with id 4536f2f237740c400d44f3274fe7bb5bca20ac0fc6649cb8d964186ed818e2fc Jun 02 15:47:05.823766 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:05.823744 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b11978_f71a_49d3_bf1e_2eb41d30143b.slice/crio-5a4220d93ef28dae7b8bb75f3a28fc3c811b5d6c8e648aef87cfaa0d9b5d8b5d WatchSource:0}: Error finding container 5a4220d93ef28dae7b8bb75f3a28fc3c811b5d6c8e648aef87cfaa0d9b5d8b5d: Status 404 returned error can't find the container with id 5a4220d93ef28dae7b8bb75f3a28fc3c811b5d6c8e648aef87cfaa0d9b5d8b5d Jun 02 15:47:05.825609 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:05.825203 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7580d5_42f8_4a02_aa56_1f5b15dff6b1.slice/crio-ba1db3b74e1879d3f12e3f5f537dbaf224a60d424fb4f782c8d9a3b4f1fd80ee WatchSource:0}: Error finding container ba1db3b74e1879d3f12e3f5f537dbaf224a60d424fb4f782c8d9a3b4f1fd80ee: Status 404 returned error can't find the container with id ba1db3b74e1879d3f12e3f5f537dbaf224a60d424fb4f782c8d9a3b4f1fd80ee Jun 02 15:47:05.826150 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:05.826100 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ff6dbe_9cfd_4e96_9b58_cc8c07878974.slice/crio-0b414ba06ade881c4fc6336325c251ef86575b26ce147e13844eee74504f8edd WatchSource:0}: Error finding container 0b414ba06ade881c4fc6336325c251ef86575b26ce147e13844eee74504f8edd: Status 404 returned error can't find the container with id 0b414ba06ade881c4fc6336325c251ef86575b26ce147e13844eee74504f8edd Jun 02 15:47:05.826798 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:05.826684 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd57594_0419_401e_8406_38e65a8e0ef5.slice/crio-522118398f102d0d787eb7a20e98ba5db8d64c5b3ba950a1a8de66a5fb639616 WatchSource:0}: Error finding container 522118398f102d0d787eb7a20e98ba5db8d64c5b3ba950a1a8de66a5fb639616: Status 404 returned error can't find the container with id 522118398f102d0d787eb7a20e98ba5db8d64c5b3ba950a1a8de66a5fb639616 Jun 02 15:47:06.553706 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.553641 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" event={"ID":"0cd57594-0419-401e-8406-38e65a8e0ef5","Type":"ContainerStarted","Data":"522118398f102d0d787eb7a20e98ba5db8d64c5b3ba950a1a8de66a5fb639616"} Jun 02 15:47:06.556914 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.556850 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb" event={"ID":"46349232-ad6b-4e6d-851e-53efe1621539","Type":"ContainerStarted","Data":"4536f2f237740c400d44f3274fe7bb5bca20ac0fc6649cb8d964186ed818e2fc"} Jun 02 15:47:06.564883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.563831 2578 generic.go:358] "Generic (PLEG): container finished" podID="e430bfcf-7490-4927-b6cc-f89f9d081f39" containerID="4ca937708a24f4861cc5e25957019b5f5a60bd7d79de0524888824f8c2a828ff" exitCode=0 Jun 02 15:47:06.564883 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.563908 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerDied","Data":"4ca937708a24f4861cc5e25957019b5f5a60bd7d79de0524888824f8c2a828ff"} Jun 02 15:47:06.568381 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.568309 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" event={"ID":"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974","Type":"ContainerStarted","Data":"0b414ba06ade881c4fc6336325c251ef86575b26ce147e13844eee74504f8edd"} Jun 02 15:47:06.570500 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.570438 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb" event={"ID":"ed7580d5-42f8-4a02-aa56-1f5b15dff6b1","Type":"ContainerStarted","Data":"ba1db3b74e1879d3f12e3f5f537dbaf224a60d424fb4f782c8d9a3b4f1fd80ee"} Jun 02 15:47:06.574185 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.574147 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" event={"ID":"87b11978-f71a-49d3-bf1e-2eb41d30143b","Type":"ContainerStarted","Data":"5a4220d93ef28dae7b8bb75f3a28fc3c811b5d6c8e648aef87cfaa0d9b5d8b5d"} Jun 02 15:47:06.576993 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.576970 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-chj4l" event={"ID":"b9b9abd4-4ecd-44ca-9329-a1923368bd46","Type":"ContainerStarted","Data":"883b0a179bad7e47501afad9b0781f922f4d521dac03e8983b8631cc8a8ec0f2"} Jun 02 15:47:06.608423 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.607764 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:47:06.615890 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.615824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8949689-7df4-4ab2-b219-e6bc3ee79e40-original-pull-secret\") pod \"global-pull-secret-syncer-5f5cp\" (UID: \"d8949689-7df4-4ab2-b219-e6bc3ee79e40\") " pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:47:06.729233 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.728869 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5f5cp" Jun 02 15:47:06.936847 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:06.936582 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5f5cp"] Jun 02 15:47:07.011354 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.011074 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:07.011354 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.011151 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:07.011354 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.011192 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:07.011354 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.011232 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.011900 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012050 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012113 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls podName:737dbd8f-f1e2-4152-90a0-c8b86b42a170 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:11.012093216 +0000 UTC m=+42.141429436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-s9svn" (UID: "737dbd8f-f1e2-4152-90a0-c8b86b42a170") : secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012560 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012616 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls podName:e1de098e-0945-4d7d-bb6e-9d97f8e47765 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:11.012599308 +0000 UTC m=+42.141935511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-4kq25" (UID: "e1de098e-0945-4d7d-bb6e-9d97f8e47765") : secret "samples-operator-tls" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012681 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012693 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84b49587fd-824gx: secret "image-registry-tls" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012724 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls podName:626b807d-b328-4e35-9d57-48fe8ecc46f4 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:11.012714008 +0000 UTC m=+42.142050212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls") pod "image-registry-84b49587fd-824gx" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4") : secret "image-registry-tls" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012776 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012803 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:11.012793941 +0000 UTC m=+42.142130145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : secret "router-metrics-certs-default" not found Jun 02 15:47:07.012943 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.012862 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:11.012851795 +0000 UTC m=+42.142188005 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : configmap references non-existent config key: service-ca.crt Jun 02 15:47:07.112872 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.112777 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:07.113046 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.112884 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:07.113046 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.113003 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:47:07.113148 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.113054 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:47:07.113148 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.113081 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls podName:dd41e666-70e5-4b55-b2d2-bcb8e142678e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:11.113058954 +0000 UTC m=+42.242395169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls") pod "dns-default-5jxnm" (UID: "dd41e666-70e5-4b55-b2d2-bcb8e142678e") : secret "dns-default-metrics-tls" not found Jun 02 15:47:07.113148 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.113104 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert podName:6dd709d8-d6a3-42aa-bef6-9fa70025b49b nodeName:}" failed. No retries permitted until 2026-06-02 15:47:11.113089174 +0000 UTC m=+42.242425390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert") pod "ingress-canary-r8wp6" (UID: "6dd709d8-d6a3-42aa-bef6-9fa70025b49b") : secret "canary-serving-cert" not found Jun 02 15:47:07.416766 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.415595 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:07.416766 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.415888 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:47:07.416766 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:07.415952 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert podName:efe6293b-b3ab-49e5-ada2-61884c0bed13 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:11.415932821 +0000 UTC m=+42.545269028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-hp7hn" (UID: "efe6293b-b3ab-49e5-ada2-61884c0bed13") : secret "networking-console-plugin-cert" not found Jun 02 15:47:07.588155 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.588081 2578 generic.go:358] "Generic (PLEG): container finished" podID="e430bfcf-7490-4927-b6cc-f89f9d081f39" containerID="57e0178e8367750baa2998d737ecaa0463e253229d252eb628df321e12f115e5" exitCode=0 Jun 02 15:47:07.588708 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.588208 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerDied","Data":"57e0178e8367750baa2998d737ecaa0463e253229d252eb628df321e12f115e5"} Jun 02 15:47:07.593128 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:07.593074 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5f5cp" event={"ID":"d8949689-7df4-4ab2-b219-e6bc3ee79e40","Type":"ContainerStarted","Data":"578861c997c843603fc14bcd031ef9fc122f3383b17f63b0ade1a04ce47a063c"} Jun 02 15:47:08.599733 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:08.599683 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" event={"ID":"e430bfcf-7490-4927-b6cc-f89f9d081f39","Type":"ContainerStarted","Data":"d0947a8e75ead0dcefbfa6cd296429098e91f6e23a434c18c1935f8d8f3be185"} Jun 02 15:47:08.624604 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:08.624199 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qlzs5" podStartSLOduration=5.828302859 podStartE2EDuration="39.624181242s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:46:32.077201056 +0000 UTC m=+3.206537271" lastFinishedPulling="2026-06-02 15:47:05.873079446 +0000 UTC m=+37.002415654" observedRunningTime="2026-06-02 15:47:08.621903796 +0000 UTC m=+39.751240020" watchObservedRunningTime="2026-06-02 15:47:08.624181242 +0000 UTC m=+39.753517468" Jun 02 15:47:11.053157 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:11.053119 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:11.053184 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:11.053228 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:11.053254 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053302 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:11.053318 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053325 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84b49587fd-824gx: secret "image-registry-tls" not found Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053300 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053405 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls podName:626b807d-b328-4e35-9d57-48fe8ecc46f4 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:19.05338553 +0000 UTC m=+50.182721731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls") pod "image-registry-84b49587fd-824gx" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4") : secret "image-registry-tls" not found Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053424 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053445 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls podName:e1de098e-0945-4d7d-bb6e-9d97f8e47765 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:19.053427064 +0000 UTC m=+50.182763361 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-4kq25" (UID: "e1de098e-0945-4d7d-bb6e-9d97f8e47765") : secret "samples-operator-tls" not found Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053463 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:19.05345372 +0000 UTC m=+50.182789924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : configmap references non-existent config key: service-ca.crt Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053496 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:19.053478931 +0000 UTC m=+50.182815138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : secret "router-metrics-certs-default" not found Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053430 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:11.053790 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.053553 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls podName:737dbd8f-f1e2-4152-90a0-c8b86b42a170 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:19.053539575 +0000 UTC m=+50.182875798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-s9svn" (UID: "737dbd8f-f1e2-4152-90a0-c8b86b42a170") : secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:11.154575 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:11.154536 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:11.154771 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.154693 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:47:11.154771 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.154768 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert podName:6dd709d8-d6a3-42aa-bef6-9fa70025b49b nodeName:}" failed. No retries permitted until 2026-06-02 15:47:19.154751214 +0000 UTC m=+50.284087415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert") pod "ingress-canary-r8wp6" (UID: "6dd709d8-d6a3-42aa-bef6-9fa70025b49b") : secret "canary-serving-cert" not found Jun 02 15:47:11.154881 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:11.154701 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:11.154881 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.154788 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:47:11.154881 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.154838 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls podName:dd41e666-70e5-4b55-b2d2-bcb8e142678e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:19.154822486 +0000 UTC m=+50.284158688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls") pod "dns-default-5jxnm" (UID: "dd41e666-70e5-4b55-b2d2-bcb8e142678e") : secret "dns-default-metrics-tls" not found Jun 02 15:47:11.457631 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:11.457593 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:11.457830 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.457738 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:47:11.457830 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:11.457802 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert podName:efe6293b-b3ab-49e5-ada2-61884c0bed13 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:19.457784463 +0000 UTC m=+50.587120671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-hp7hn" (UID: "efe6293b-b3ab-49e5-ada2-61884c0bed13") : secret "networking-console-plugin-cert" not found Jun 02 15:47:14.614835 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.614515 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-chj4l" event={"ID":"b9b9abd4-4ecd-44ca-9329-a1923368bd46","Type":"ContainerStarted","Data":"577f79d4af84da7fee153f2e0086734f1ab91cc688eb19080bf6405edd182b6d"} Jun 02 15:47:14.617259 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.616785 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" event={"ID":"0cd57594-0419-401e-8406-38e65a8e0ef5","Type":"ContainerStarted","Data":"77cf63201ef18e2df0f566f4ef168a232409409ccaee0c62ecc4566975e84fb7"} Jun 02 15:47:14.623317 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.622839 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb" event={"ID":"46349232-ad6b-4e6d-851e-53efe1621539","Type":"ContainerStarted","Data":"952231cc2cf6fd006c18f84001bb7f9c43f72cb5ed49dad08982e0f282a60d57"} Jun 02 15:47:14.628683 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.628011 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/0.log" Jun 02 15:47:14.628683 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.628047 2578 generic.go:358] "Generic (PLEG): container finished" podID="e3ff6dbe-9cfd-4e96-9b58-cc8c07878974" containerID="8fa4ca49ce982aa57dd4cd3bb428f6fb5a8af587d4827b62f4305ff0c1507097" exitCode=255 Jun 02 15:47:14.628683 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.628105 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" event={"ID":"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974","Type":"ContainerDied","Data":"8fa4ca49ce982aa57dd4cd3bb428f6fb5a8af587d4827b62f4305ff0c1507097"} Jun 02 15:47:14.628683 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.628410 2578 scope.go:117] "RemoveContainer" containerID="8fa4ca49ce982aa57dd4cd3bb428f6fb5a8af587d4827b62f4305ff0c1507097" Jun 02 15:47:14.630623 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.630215 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5f5cp" event={"ID":"d8949689-7df4-4ab2-b219-e6bc3ee79e40","Type":"ContainerStarted","Data":"5bc4248a816821a076b75150e7ab01e10bd2542c16c2ad9fd47dccccc05c375a"} Jun 02 15:47:14.633321 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.633249 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb" event={"ID":"ed7580d5-42f8-4a02-aa56-1f5b15dff6b1","Type":"ContainerStarted","Data":"968f0963932f9d1b47ff2e07cf7d0917bb8b6e091078d66cae6a5b879a8892cd"} Jun 02 15:47:14.635537 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.635429 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" event={"ID":"87b11978-f71a-49d3-bf1e-2eb41d30143b","Type":"ContainerStarted","Data":"7d06d3e52508699e6d1a1be6e3c3288bc5cc07214e6dad8d44c24b639769cd5a"} Jun 02 15:47:14.648490 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.648181 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-788bf58df5-chj4l" podStartSLOduration=11.209801995 podStartE2EDuration="19.648162365s" podCreationTimestamp="2026-06-02 15:46:55 +0000 UTC" firstStartedPulling="2026-06-02 15:47:05.816342943 +0000 UTC m=+36.945679144" lastFinishedPulling="2026-06-02 15:47:14.254703302 +0000 UTC m=+45.384039514" observedRunningTime="2026-06-02 15:47:14.646333035 +0000 UTC m=+45.775669260" watchObservedRunningTime="2026-06-02 15:47:14.648162365 +0000 UTC m=+45.777498588" Jun 02 15:47:14.678645 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.678552 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" podStartSLOduration=11.26296842 podStartE2EDuration="19.678533454s" podCreationTimestamp="2026-06-02 15:46:55 +0000 UTC" firstStartedPulling="2026-06-02 15:47:05.82624413 +0000 UTC m=+36.955580349" lastFinishedPulling="2026-06-02 15:47:14.241809175 +0000 UTC m=+45.371145383" observedRunningTime="2026-06-02 15:47:14.676344285 +0000 UTC m=+45.805680506" watchObservedRunningTime="2026-06-02 15:47:14.678533454 +0000 UTC m=+45.807869679" Jun 02 15:47:14.731917 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.731577 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-77fmb" podStartSLOduration=11.339418003 podStartE2EDuration="19.731556699s" podCreationTimestamp="2026-06-02 15:46:55 +0000 UTC" firstStartedPulling="2026-06-02 15:47:05.849667526 +0000 UTC m=+36.979003728" lastFinishedPulling="2026-06-02 15:47:14.241806207 +0000 UTC m=+45.371142424" observedRunningTime="2026-06-02 15:47:14.728320577 +0000 UTC m=+45.857656801" watchObservedRunningTime="2026-06-02 15:47:14.731556699 +0000 UTC m=+45.860892916" Jun 02 15:47:14.733231 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.733116 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" podStartSLOduration=11.328221038 podStartE2EDuration="19.733099748s" podCreationTimestamp="2026-06-02 15:46:55 +0000 UTC" firstStartedPulling="2026-06-02 15:47:05.849783242 +0000 UTC m=+36.979119452" lastFinishedPulling="2026-06-02 15:47:14.254661952 +0000 UTC m=+45.383998162" observedRunningTime="2026-06-02 15:47:14.701979023 +0000 UTC m=+45.831315247" watchObservedRunningTime="2026-06-02 15:47:14.733099748 +0000 UTC m=+45.862435972" Jun 02 15:47:14.805963 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.805905 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5f5cp" podStartSLOduration=33.349910274 podStartE2EDuration="40.805886583s" podCreationTimestamp="2026-06-02 15:46:34 +0000 UTC" firstStartedPulling="2026-06-02 15:47:06.942772637 +0000 UTC m=+38.072108843" lastFinishedPulling="2026-06-02 15:47:14.398748945 +0000 UTC m=+45.528085152" observedRunningTime="2026-06-02 15:47:14.804596778 +0000 UTC m=+45.933933003" watchObservedRunningTime="2026-06-02 15:47:14.805886583 +0000 UTC m=+45.935222804" Jun 02 15:47:14.825635 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:14.825576 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-98rxb" podStartSLOduration=8.38710011 podStartE2EDuration="16.825562127s" podCreationTimestamp="2026-06-02 15:46:58 +0000 UTC" firstStartedPulling="2026-06-02 15:47:05.825462962 +0000 UTC m=+36.954799166" lastFinishedPulling="2026-06-02 15:47:14.263924979 +0000 UTC m=+45.393261183" observedRunningTime="2026-06-02 15:47:14.824682014 +0000 UTC m=+45.954018239" watchObservedRunningTime="2026-06-02 15:47:14.825562127 +0000 UTC m=+45.954898355" Jun 02 15:47:15.640040 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:15.640008 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 15:47:15.640494 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:15.640393 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/0.log" Jun 02 15:47:15.640494 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:15.640430 2578 generic.go:358] "Generic (PLEG): container finished" podID="e3ff6dbe-9cfd-4e96-9b58-cc8c07878974" containerID="1a2bdf05c910578337dc8e250b1d8bcbec0165958951c9909e956bf1fc361658" exitCode=255 Jun 02 15:47:15.640593 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:15.640535 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" event={"ID":"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974","Type":"ContainerDied","Data":"1a2bdf05c910578337dc8e250b1d8bcbec0165958951c9909e956bf1fc361658"} Jun 02 15:47:15.640593 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:15.640578 2578 scope.go:117] "RemoveContainer" containerID="8fa4ca49ce982aa57dd4cd3bb428f6fb5a8af587d4827b62f4305ff0c1507097" Jun 02 15:47:15.640808 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:15.640786 2578 scope.go:117] "RemoveContainer" containerID="1a2bdf05c910578337dc8e250b1d8bcbec0165958951c9909e956bf1fc361658" Jun 02 15:47:15.641015 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:15.640989 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-khtbj_openshift-console-operator(e3ff6dbe-9cfd-4e96-9b58-cc8c07878974)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" podUID="e3ff6dbe-9cfd-4e96-9b58-cc8c07878974" Jun 02 15:47:16.644967 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:16.644936 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 15:47:16.645447 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:16.645326 2578 scope.go:117] "RemoveContainer" containerID="1a2bdf05c910578337dc8e250b1d8bcbec0165958951c9909e956bf1fc361658" Jun 02 15:47:16.645588 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:16.645567 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-khtbj_openshift-console-operator(e3ff6dbe-9cfd-4e96-9b58-cc8c07878974)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" podUID="e3ff6dbe-9cfd-4e96-9b58-cc8c07878974" Jun 02 15:47:17.502380 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:17.502349 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q4c6g_2de345a5-85e0-422e-8497-e9d7c2fbdcc4/dns-node-resolver/0.log" Jun 02 15:47:18.303625 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:18.303598 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5ptj6_d794b47c-d098-448e-926e-1e81d554e31f/node-ca/0.log" Jun 02 15:47:19.130437 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:19.130397 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:19.130453 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:19.130483 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:19.130508 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130535 2578 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130558 2578 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84b49587fd-824gx: secret "image-registry-tls" not found Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:19.130571 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130618 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls podName:626b807d-b328-4e35-9d57-48fe8ecc46f4 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.130602758 +0000 UTC m=+66.259938959 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls") pod "image-registry-84b49587fd-824gx" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4") : secret "image-registry-tls" not found Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130621 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130659 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130621 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130686 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.130669534 +0000 UTC m=+66.260005753 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : secret "router-metrics-certs-default" not found Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130729 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle podName:2b93a739-0f4e-4492-9616-9f2ae397ecd0 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.130716146 +0000 UTC m=+66.260052350 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle") pod "router-default-6985979865-kdkr9" (UID: "2b93a739-0f4e-4492-9616-9f2ae397ecd0") : configmap references non-existent config key: service-ca.crt Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130752 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls podName:737dbd8f-f1e2-4152-90a0-c8b86b42a170 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.130744823 +0000 UTC m=+66.260081024 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-69b6cbdb88-s9svn" (UID: "737dbd8f-f1e2-4152-90a0-c8b86b42a170") : secret "cluster-monitoring-operator-tls" not found Jun 02 15:47:19.130796 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.130763 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls podName:e1de098e-0945-4d7d-bb6e-9d97f8e47765 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.130757041 +0000 UTC m=+66.260093242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-4kq25" (UID: "e1de098e-0945-4d7d-bb6e-9d97f8e47765") : secret "samples-operator-tls" not found Jun 02 15:47:19.231027 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:19.230994 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:19.231204 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:19.231144 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:19.231204 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.231146 2578 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:47:19.231204 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.231201 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:47:19.231330 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.231261 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls podName:dd41e666-70e5-4b55-b2d2-bcb8e142678e nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.231239721 +0000 UTC m=+66.360575929 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls") pod "dns-default-5jxnm" (UID: "dd41e666-70e5-4b55-b2d2-bcb8e142678e") : secret "dns-default-metrics-tls" not found Jun 02 15:47:19.231330 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.231277 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert podName:6dd709d8-d6a3-42aa-bef6-9fa70025b49b nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.231269042 +0000 UTC m=+66.360605243 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert") pod "ingress-canary-r8wp6" (UID: "6dd709d8-d6a3-42aa-bef6-9fa70025b49b") : secret "canary-serving-cert" not found Jun 02 15:47:19.533149 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:19.533045 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:19.533554 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.533202 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:47:19.533554 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:19.533282 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert podName:efe6293b-b3ab-49e5-ada2-61884c0bed13 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:35.533265151 +0000 UTC m=+66.662601352 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-hp7hn" (UID: "efe6293b-b3ab-49e5-ada2-61884c0bed13") : secret "networking-console-plugin-cert" not found Jun 02 15:47:23.630468 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:23.630424 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:23.630468 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:23.630469 2578 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:23.630900 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:23.630851 2578 scope.go:117] "RemoveContainer" containerID="1a2bdf05c910578337dc8e250b1d8bcbec0165958951c9909e956bf1fc361658" Jun 02 15:47:23.631042 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:23.631025 2578 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-khtbj_openshift-console-operator(e3ff6dbe-9cfd-4e96-9b58-cc8c07878974)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" podUID="e3ff6dbe-9cfd-4e96-9b58-cc8c07878974" Jun 02 15:47:29.554718 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:29.554689 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnkrg" Jun 02 15:47:35.169553 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.169514 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:35.170071 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.169587 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:35.170071 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.169621 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:47:35.170071 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.169655 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:35.170071 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.169807 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:35.170071 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.169865 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:35.170071 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.169895 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:47:35.170759 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.170731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b93a739-0f4e-4492-9616-9f2ae397ecd0-service-ca-bundle\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:35.172653 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.172629 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 15:47:35.173825 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.173800 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"image-registry-84b49587fd-824gx\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:35.173926 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.173821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/737dbd8f-f1e2-4152-90a0-c8b86b42a170-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-69b6cbdb88-s9svn\" (UID: \"737dbd8f-f1e2-4152-90a0-c8b86b42a170\") " pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:35.173926 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.173801 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e1de098e-0945-4d7d-bb6e-9d97f8e47765-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-4kq25\" (UID: \"e1de098e-0945-4d7d-bb6e-9d97f8e47765\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:35.173926 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.173821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b93a739-0f4e-4492-9616-9f2ae397ecd0-metrics-certs\") pod \"router-default-6985979865-kdkr9\" (UID: \"2b93a739-0f4e-4492-9616-9f2ae397ecd0\") " pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:35.174030 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.173980 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9tqr\" (UniqueName: \"kubernetes.io/projected/3a94647c-a5b5-4c9d-a13d-d6c824a3a79e-kube-api-access-w9tqr\") pod \"network-check-target-94lpx\" (UID: \"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e\") " pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:47:35.182496 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.182477 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877796b1-2b26-4c87-903a-1235f441e838-metrics-certs\") pod \"network-metrics-daemon-7wvwq\" (UID: \"877796b1-2b26-4c87-903a-1235f441e838\") " pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:47:35.225014 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.224989 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6qg5h\"" Jun 02 15:47:35.232763 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.232740 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:47:35.237889 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.237870 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2vmzq\"" Jun 02 15:47:35.245732 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.245712 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7wvwq" Jun 02 15:47:35.270941 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.270898 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:35.271110 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.270973 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:35.273318 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.273291 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd41e666-70e5-4b55-b2d2-bcb8e142678e-metrics-tls\") pod \"dns-default-5jxnm\" (UID: \"dd41e666-70e5-4b55-b2d2-bcb8e142678e\") " pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:35.273582 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.273565 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dd709d8-d6a3-42aa-bef6-9fa70025b49b-cert\") pod \"ingress-canary-r8wp6\" (UID: \"6dd709d8-d6a3-42aa-bef6-9fa70025b49b\") " pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:35.354269 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.354233 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vhj6z\"" Jun 02 15:47:35.362278 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.362252 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:35.363537 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.363514 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wp9pm\"" Jun 02 15:47:35.370440 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.370389 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-94lpx"] Jun 02 15:47:35.371276 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.371255 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" Jun 02 15:47:35.373306 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.373288 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-x2kvd\"" Jun 02 15:47:35.376711 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:35.376688 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a94647c_a5b5_4c9d_a13d_d6c824a3a79e.slice/crio-7adc47d9131aee15ea27cf99629bc761f08e3a737127c08581abce26c3105284 WatchSource:0}: Error finding container 7adc47d9131aee15ea27cf99629bc761f08e3a737127c08581abce26c3105284: Status 404 returned error can't find the container with id 7adc47d9131aee15ea27cf99629bc761f08e3a737127c08581abce26c3105284 Jun 02 15:47:35.381200 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.381177 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:35.394341 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.394258 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7wvwq"] Jun 02 15:47:35.412950 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.412020 2578 scope.go:117] "RemoveContainer" containerID="1a2bdf05c910578337dc8e250b1d8bcbec0165958951c9909e956bf1fc361658" Jun 02 15:47:35.415403 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.415190 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vxmkc\"" Jun 02 15:47:35.421680 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.421340 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" Jun 02 15:47:35.447503 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.446759 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ws8nx\"" Jun 02 15:47:35.451907 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.451859 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g6cp8\"" Jun 02 15:47:35.455701 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.454793 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r8wp6" Jun 02 15:47:35.458912 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.458890 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:35.557037 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.556940 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84b49587fd-824gx"] Jun 02 15:47:35.561909 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:35.561850 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626b807d_b328_4e35_9d57_48fe8ecc46f4.slice/crio-60895adf142c3f97f7addb4050fb6ce5829eb9b754688d049a28b8d200235273 WatchSource:0}: Error finding container 60895adf142c3f97f7addb4050fb6ce5829eb9b754688d049a28b8d200235273: Status 404 returned error can't find the container with id 60895adf142c3f97f7addb4050fb6ce5829eb9b754688d049a28b8d200235273 Jun 02 15:47:35.573355 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.572844 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:35.584557 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.584096 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/efe6293b-b3ab-49e5-ada2-61884c0bed13-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-hp7hn\" (UID: \"efe6293b-b3ab-49e5-ada2-61884c0bed13\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:35.598095 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.594052 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25"] Jun 02 15:47:35.634959 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.634930 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6985979865-kdkr9"] Jun 02 15:47:35.642045 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:35.640500 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b93a739_0f4e_4492_9616_9f2ae397ecd0.slice/crio-cae84ac59fe10e7c5050764eb8b3fe09e4aa8baad88b65c3c4382d187a10a360 WatchSource:0}: Error finding container cae84ac59fe10e7c5050764eb8b3fe09e4aa8baad88b65c3c4382d187a10a360: Status 404 returned error can't find the container with id cae84ac59fe10e7c5050764eb8b3fe09e4aa8baad88b65c3c4382d187a10a360 Jun 02 15:47:35.685011 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.684963 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn"] Jun 02 15:47:35.688836 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:35.688812 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737dbd8f_f1e2_4152_90a0_c8b86b42a170.slice/crio-5f64bb19cdde5cf6e09413f6e8372aa1393030984e3a8eedec32e9b7df163d5b WatchSource:0}: Error finding container 5f64bb19cdde5cf6e09413f6e8372aa1393030984e3a8eedec32e9b7df163d5b: Status 404 returned error can't find the container with id 5f64bb19cdde5cf6e09413f6e8372aa1393030984e3a8eedec32e9b7df163d5b Jun 02 15:47:35.695797 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.695743 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6985979865-kdkr9" event={"ID":"2b93a739-0f4e-4492-9616-9f2ae397ecd0","Type":"ContainerStarted","Data":"cae84ac59fe10e7c5050764eb8b3fe09e4aa8baad88b65c3c4382d187a10a360"} Jun 02 15:47:35.698320 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.697656 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84b49587fd-824gx" event={"ID":"626b807d-b328-4e35-9d57-48fe8ecc46f4","Type":"ContainerStarted","Data":"b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e"} Jun 02 15:47:35.698320 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.697690 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84b49587fd-824gx" event={"ID":"626b807d-b328-4e35-9d57-48fe8ecc46f4","Type":"ContainerStarted","Data":"60895adf142c3f97f7addb4050fb6ce5829eb9b754688d049a28b8d200235273"} Jun 02 15:47:35.698320 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.698309 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:47:35.699572 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.699548 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7wvwq" event={"ID":"877796b1-2b26-4c87-903a-1235f441e838","Type":"ContainerStarted","Data":"c22813d32494bc90f1ccbbb8ee5dec24b15b015bc5e69daa5ec6058c8934e7ae"} Jun 02 15:47:35.700728 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.700706 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" event={"ID":"e1de098e-0945-4d7d-bb6e-9d97f8e47765","Type":"ContainerStarted","Data":"2ce6220b368b153d484f57cfd7f69707730a9aa93152065d6fe5c638c0eb8b3a"} Jun 02 15:47:35.702589 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.702572 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 15:47:35.702656 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.702643 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" event={"ID":"e3ff6dbe-9cfd-4e96-9b58-cc8c07878974","Type":"ContainerStarted","Data":"f3758d21f762287e9aea67074a38952b85fd30bec7073b16c763749642e00e39"} Jun 02 15:47:35.703317 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.703296 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:35.704621 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.704589 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" event={"ID":"737dbd8f-f1e2-4152-90a0-c8b86b42a170","Type":"ContainerStarted","Data":"5f64bb19cdde5cf6e09413f6e8372aa1393030984e3a8eedec32e9b7df163d5b"} Jun 02 15:47:35.707778 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.707746 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-94lpx" event={"ID":"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e","Type":"ContainerStarted","Data":"844e5f29bdde88ed448fd9503c3b1b80c9be9f2f24ba04c3efa3621619d7c8c2"} Jun 02 15:47:35.707876 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.707805 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-94lpx" event={"ID":"3a94647c-a5b5-4c9d-a13d-d6c824a3a79e","Type":"ContainerStarted","Data":"7adc47d9131aee15ea27cf99629bc761f08e3a737127c08581abce26c3105284"} Jun 02 15:47:35.707950 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.707874 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:47:35.708396 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.708356 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5jxnm"] Jun 02 15:47:35.711002 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:35.710979 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd41e666_70e5_4b55_b2d2_bcb8e142678e.slice/crio-a8e1b868d246bda9d4367aa2d6d0b29ed3086a8f69f6af1e41ad74063ef7afe3 WatchSource:0}: Error finding container a8e1b868d246bda9d4367aa2d6d0b29ed3086a8f69f6af1e41ad74063ef7afe3: Status 404 returned error can't find the container with id a8e1b868d246bda9d4367aa2d6d0b29ed3086a8f69f6af1e41ad74063ef7afe3 Jun 02 15:47:35.720742 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.720719 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r8wp6"] Jun 02 15:47:35.723307 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:35.723279 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd709d8_d6a3_42aa_bef6_9fa70025b49b.slice/crio-eff30ab6e79d10bb5fddc90a8debe5918f37fb0440360b91fbac472fcc4d106f WatchSource:0}: Error finding container eff30ab6e79d10bb5fddc90a8debe5918f37fb0440360b91fbac472fcc4d106f: Status 404 returned error can't find the container with id eff30ab6e79d10bb5fddc90a8debe5918f37fb0440360b91fbac472fcc4d106f Jun 02 15:47:35.725435 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.725287 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84b49587fd-824gx" podStartSLOduration=65.725272177 podStartE2EDuration="1m5.725272177s" podCreationTimestamp="2026-06-02 15:46:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:47:35.724267558 +0000 UTC m=+66.853603782" watchObservedRunningTime="2026-06-02 15:47:35.725272177 +0000 UTC m=+66.854608402" Jun 02 15:47:35.740423 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.740338 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-94lpx" podStartSLOduration=66.740319756 podStartE2EDuration="1m6.740319756s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:47:35.739654203 +0000 UTC m=+66.868990425" watchObservedRunningTime="2026-06-02 15:47:35.740319756 +0000 UTC m=+66.869655980" Jun 02 15:47:35.761411 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.761359 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-slmnc\"" Jun 02 15:47:35.763201 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.763159 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" podStartSLOduration=32.357796882 podStartE2EDuration="40.763146094s" podCreationTimestamp="2026-06-02 15:46:55 +0000 UTC" firstStartedPulling="2026-06-02 15:47:05.849830235 +0000 UTC m=+36.979166443" lastFinishedPulling="2026-06-02 15:47:14.255179451 +0000 UTC m=+45.384515655" observedRunningTime="2026-06-02 15:47:35.762307688 +0000 UTC m=+66.891643909" watchObservedRunningTime="2026-06-02 15:47:35.763146094 +0000 UTC m=+66.892482317" Jun 02 15:47:35.768402 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.768377 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" Jun 02 15:47:35.918272 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:35.918214 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn"] Jun 02 15:47:35.922351 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:35.922175 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe6293b_b3ab_49e5_ada2_61884c0bed13.slice/crio-d809e38e656f43f805f391a29cac1d171e6d07e64f5f5d56de1cce330613a6b2 WatchSource:0}: Error finding container d809e38e656f43f805f391a29cac1d171e6d07e64f5f5d56de1cce330613a6b2: Status 404 returned error can't find the container with id d809e38e656f43f805f391a29cac1d171e6d07e64f5f5d56de1cce330613a6b2 Jun 02 15:47:36.256892 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:36.256860 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-khtbj" Jun 02 15:47:36.719055 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:36.719017 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6985979865-kdkr9" event={"ID":"2b93a739-0f4e-4492-9616-9f2ae397ecd0","Type":"ContainerStarted","Data":"3096207ba7104fd6869145032ea2ebab0a28771a2bdb1f55c366c22b1a7d7ce0"} Jun 02 15:47:36.723312 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:36.723184 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" event={"ID":"efe6293b-b3ab-49e5-ada2-61884c0bed13","Type":"ContainerStarted","Data":"d809e38e656f43f805f391a29cac1d171e6d07e64f5f5d56de1cce330613a6b2"} Jun 02 15:47:36.726568 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:36.726516 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5jxnm" event={"ID":"dd41e666-70e5-4b55-b2d2-bcb8e142678e","Type":"ContainerStarted","Data":"a8e1b868d246bda9d4367aa2d6d0b29ed3086a8f69f6af1e41ad74063ef7afe3"} Jun 02 15:47:36.731984 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:36.731954 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r8wp6" event={"ID":"6dd709d8-d6a3-42aa-bef6-9fa70025b49b","Type":"ContainerStarted","Data":"eff30ab6e79d10bb5fddc90a8debe5918f37fb0440360b91fbac472fcc4d106f"} Jun 02 15:47:37.381578 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:37.381525 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:37.384910 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:37.384880 2578 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:37.413138 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:37.413073 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-6985979865-kdkr9" podStartSLOduration=42.413054874 podStartE2EDuration="42.413054874s" podCreationTimestamp="2026-06-02 15:46:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:47:36.746716471 +0000 UTC m=+67.876052709" watchObservedRunningTime="2026-06-02 15:47:37.413054874 +0000 UTC m=+68.542391100" Jun 02 15:47:37.739665 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:37.739567 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7wvwq" event={"ID":"877796b1-2b26-4c87-903a-1235f441e838","Type":"ContainerStarted","Data":"dc6e12394d2b7fe2ddd503f4ae0080be8dedb8963adabf6dce396e139f75f927"} Jun 02 15:47:37.739925 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:37.739901 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:37.741191 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:37.741164 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6985979865-kdkr9" Jun 02 15:47:38.636356 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.636316 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m8ftl"] Jun 02 15:47:38.640706 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.640675 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.645178 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.645156 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tl6r5\"" Jun 02 15:47:38.645949 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.645928 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 15:47:38.645949 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.645942 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 15:47:38.657245 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.657222 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m8ftl"] Jun 02 15:47:38.702476 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.702435 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.702640 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.702540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjdrs\" (UniqueName: \"kubernetes.io/projected/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-kube-api-access-rjdrs\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.702640 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.702577 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-data-volume\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.702640 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.702623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-crio-socket\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.702812 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.702699 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.803750 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.803710 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rjdrs\" (UniqueName: \"kubernetes.io/projected/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-kube-api-access-rjdrs\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.803909 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.803766 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-data-volume\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.803977 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.803943 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-crio-socket\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.804357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.804093 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.804357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.804106 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-data-volume\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.804357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.804172 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.804357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.804200 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-crio-socket\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.804666 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.804657 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.806814 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.806787 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.814357 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.814332 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjdrs\" (UniqueName: \"kubernetes.io/projected/141ed18e-35e0-408a-aaeb-e68b1c5c4e21-kube-api-access-rjdrs\") pod \"insights-runtime-extractor-m8ftl\" (UID: \"141ed18e-35e0-408a-aaeb-e68b1c5c4e21\") " pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:38.951080 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:38.950985 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m8ftl" Jun 02 15:47:39.442984 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.440194 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m8ftl"] Jun 02 15:47:39.748839 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.748762 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" event={"ID":"efe6293b-b3ab-49e5-ada2-61884c0bed13","Type":"ContainerStarted","Data":"aeb675ba0c54837e4d22bec786d21f131e880910c5f22805c795bac02338f0ba"} Jun 02 15:47:39.750424 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.750394 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5jxnm" event={"ID":"dd41e666-70e5-4b55-b2d2-bcb8e142678e","Type":"ContainerStarted","Data":"bfa8af5f92a8ac354518f99597c61b4956208031474ffc53e9a6c29f0f192b56"} Jun 02 15:47:39.750554 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.750430 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5jxnm" event={"ID":"dd41e666-70e5-4b55-b2d2-bcb8e142678e","Type":"ContainerStarted","Data":"917581fcdc64f5aaba1c1584ca0095d74c1818ac56000b3ae89812b012007a5b"} Jun 02 15:47:39.750554 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.750517 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:39.755895 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.755864 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7wvwq" event={"ID":"877796b1-2b26-4c87-903a-1235f441e838","Type":"ContainerStarted","Data":"6d0fbbc4955d0799907309a2095d1d8d6ed391d5ec9e1688349d01885b81f46e"} Jun 02 15:47:39.757208 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.757184 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8ftl" event={"ID":"141ed18e-35e0-408a-aaeb-e68b1c5c4e21","Type":"ContainerStarted","Data":"7522f6c330c91373aed29c113c52dd615cf130b1cb356a4085262d70930b3e72"} Jun 02 15:47:39.757316 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.757213 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8ftl" event={"ID":"141ed18e-35e0-408a-aaeb-e68b1c5c4e21","Type":"ContainerStarted","Data":"b749d7c2ead13d36dbcabb96b2ae1134c17464cb39c12d5eccc3624d5063da20"} Jun 02 15:47:39.758610 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.758591 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" event={"ID":"e1de098e-0945-4d7d-bb6e-9d97f8e47765","Type":"ContainerStarted","Data":"9e76d156e58f9d86dbed91b6fb2c461c50af09c1d4010e2881b59dd0d9c3e904"} Jun 02 15:47:39.758610 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.758611 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" event={"ID":"e1de098e-0945-4d7d-bb6e-9d97f8e47765","Type":"ContainerStarted","Data":"fe2d0f3dc155b9c0a9d77a4cfeea3df9f175cc0ba3c9e3c2fd271b2d4e8784af"} Jun 02 15:47:39.759850 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.759827 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r8wp6" event={"ID":"6dd709d8-d6a3-42aa-bef6-9fa70025b49b","Type":"ContainerStarted","Data":"8a27848234b85a70dc9d78a0ca000f2c1950845b9b3143bde4a3565c9bffba94"} Jun 02 15:47:39.761091 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.761073 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" event={"ID":"737dbd8f-f1e2-4152-90a0-c8b86b42a170","Type":"ContainerStarted","Data":"613616244b7289983ca5908f0957450397a786e015728e6f5c62ec4496967137"} Jun 02 15:47:39.770696 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.770658 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-hp7hn" podStartSLOduration=33.449755539 podStartE2EDuration="36.770642907s" podCreationTimestamp="2026-06-02 15:47:03 +0000 UTC" firstStartedPulling="2026-06-02 15:47:35.924465292 +0000 UTC m=+67.053801493" lastFinishedPulling="2026-06-02 15:47:39.245352657 +0000 UTC m=+70.374688861" observedRunningTime="2026-06-02 15:47:39.769785144 +0000 UTC m=+70.899121367" watchObservedRunningTime="2026-06-02 15:47:39.770642907 +0000 UTC m=+70.899979130" Jun 02 15:47:39.789052 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.789003 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-4kq25" podStartSLOduration=41.209706721 podStartE2EDuration="44.788989019s" podCreationTimestamp="2026-06-02 15:46:55 +0000 UTC" firstStartedPulling="2026-06-02 15:47:35.667896183 +0000 UTC m=+66.797232385" lastFinishedPulling="2026-06-02 15:47:39.24717847 +0000 UTC m=+70.376514683" observedRunningTime="2026-06-02 15:47:39.787906189 +0000 UTC m=+70.917242414" watchObservedRunningTime="2026-06-02 15:47:39.788989019 +0000 UTC m=+70.918325248" Jun 02 15:47:39.818518 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.818462 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-5jxnm" podStartSLOduration=33.2861544 podStartE2EDuration="36.818354768s" podCreationTimestamp="2026-06-02 15:47:03 +0000 UTC" firstStartedPulling="2026-06-02 15:47:35.712731793 +0000 UTC m=+66.842067994" lastFinishedPulling="2026-06-02 15:47:39.24493215 +0000 UTC m=+70.374268362" observedRunningTime="2026-06-02 15:47:39.816687397 +0000 UTC m=+70.946023621" watchObservedRunningTime="2026-06-02 15:47:39.818354768 +0000 UTC m=+70.947690992" Jun 02 15:47:39.838641 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.838590 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r8wp6" podStartSLOduration=33.319309179 podStartE2EDuration="36.838578194s" podCreationTimestamp="2026-06-02 15:47:03 +0000 UTC" firstStartedPulling="2026-06-02 15:47:35.725880853 +0000 UTC m=+66.855217056" lastFinishedPulling="2026-06-02 15:47:39.245149858 +0000 UTC m=+70.374486071" observedRunningTime="2026-06-02 15:47:39.837883566 +0000 UTC m=+70.967219789" watchObservedRunningTime="2026-06-02 15:47:39.838578194 +0000 UTC m=+70.967914416" Jun 02 15:47:39.866528 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.866474 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7wvwq" podStartSLOduration=69.313298557 podStartE2EDuration="1m10.86645841s" podCreationTimestamp="2026-06-02 15:46:29 +0000 UTC" firstStartedPulling="2026-06-02 15:47:35.403317587 +0000 UTC m=+66.532653787" lastFinishedPulling="2026-06-02 15:47:36.956477424 +0000 UTC m=+68.085813640" observedRunningTime="2026-06-02 15:47:39.864674738 +0000 UTC m=+70.994010961" watchObservedRunningTime="2026-06-02 15:47:39.86645841 +0000 UTC m=+70.995794632" Jun 02 15:47:39.884246 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:39.884151 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-69b6cbdb88-s9svn" podStartSLOduration=41.330349232 podStartE2EDuration="44.884134686s" podCreationTimestamp="2026-06-02 15:46:55 +0000 UTC" firstStartedPulling="2026-06-02 15:47:35.691130758 +0000 UTC m=+66.820466973" lastFinishedPulling="2026-06-02 15:47:39.244916214 +0000 UTC m=+70.374252427" observedRunningTime="2026-06-02 15:47:39.883941602 +0000 UTC m=+71.013277826" watchObservedRunningTime="2026-06-02 15:47:39.884134686 +0000 UTC m=+71.013470911" Jun 02 15:47:40.766481 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:40.766438 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8ftl" event={"ID":"141ed18e-35e0-408a-aaeb-e68b1c5c4e21","Type":"ContainerStarted","Data":"4071ac045d01d7a832e4bb23d9fb5e957c6a54ce771a507f01720128f4ae709f"} Jun 02 15:47:41.771337 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:41.771251 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8ftl" event={"ID":"141ed18e-35e0-408a-aaeb-e68b1c5c4e21","Type":"ContainerStarted","Data":"c213856c10f3dc67a7a47d2b7736468453c0b4e8987b2c26c0600bcb43698242"} Jun 02 15:47:41.794468 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:41.794413 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m8ftl" podStartSLOduration=1.861609244 podStartE2EDuration="3.794396613s" podCreationTimestamp="2026-06-02 15:47:38 +0000 UTC" firstStartedPulling="2026-06-02 15:47:39.55516384 +0000 UTC m=+70.684500048" lastFinishedPulling="2026-06-02 15:47:41.487951211 +0000 UTC m=+72.617287417" observedRunningTime="2026-06-02 15:47:41.79360015 +0000 UTC m=+72.922936372" watchObservedRunningTime="2026-06-02 15:47:41.794396613 +0000 UTC m=+72.923732839" Jun 02 15:47:49.713111 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.712986 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7g4xl"] Jun 02 15:47:49.715486 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.715469 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.717591 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.717557 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 15:47:49.717591 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.717570 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 15:47:49.717772 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.717595 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 15:47:49.717772 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.717601 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 15:47:49.717772 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.717651 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zfv5r\"" Jun 02 15:47:49.769529 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.769499 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5jxnm" Jun 02 15:47:49.801211 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-sys\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.801392 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801218 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.801392 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-tls\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.801392 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801333 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-textfile\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.801392 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801351 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-metrics-client-ca\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.801530 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801444 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-wtmp\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.801530 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801468 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.801530 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-root\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.801530 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.801508 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv6wd\" (UniqueName: \"kubernetes.io/projected/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-kube-api-access-kv6wd\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.902515 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902473 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-sys\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.902702 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902534 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.902702 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902486 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-sys\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.902702 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902596 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-tls\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.902702 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902646 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-textfile\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.902702 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:49.902690 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 15:47:49.902942 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:47:49.902780 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-tls podName:a2059bfa-367d-4a5c-9d2e-8662fa11a3cc nodeName:}" failed. No retries permitted until 2026-06-02 15:47:50.402757442 +0000 UTC m=+81.532093646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-tls") pod "node-exporter-7g4xl" (UID: "a2059bfa-367d-4a5c-9d2e-8662fa11a3cc") : secret "node-exporter-tls" not found Jun 02 15:47:49.902942 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902807 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-metrics-client-ca\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.902942 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902889 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-wtmp\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.902942 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902917 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.903148 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902944 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-textfile\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.903148 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902960 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-root\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.903148 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.902999 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kv6wd\" (UniqueName: \"kubernetes.io/projected/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-kube-api-access-kv6wd\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.903148 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.903003 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-root\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.903148 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.903116 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-wtmp\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.903343 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.903224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.903343 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.903261 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-metrics-client-ca\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.905265 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.905247 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:49.924457 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:49.924433 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv6wd\" (UniqueName: \"kubernetes.io/projected/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-kube-api-access-kv6wd\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:50.407820 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:50.407782 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-tls\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:50.410002 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:50.409972 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a2059bfa-367d-4a5c-9d2e-8662fa11a3cc-node-exporter-tls\") pod \"node-exporter-7g4xl\" (UID: \"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc\") " pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:50.624987 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:50.624954 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7g4xl" Jun 02 15:47:50.633548 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:47:50.633508 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2059bfa_367d_4a5c_9d2e_8662fa11a3cc.slice/crio-96e1825ef9b3ed58235fb4b4d8c1af5b76b9a4f4c633f5282af842da2d9a4df4 WatchSource:0}: Error finding container 96e1825ef9b3ed58235fb4b4d8c1af5b76b9a4f4c633f5282af842da2d9a4df4: Status 404 returned error can't find the container with id 96e1825ef9b3ed58235fb4b4d8c1af5b76b9a4f4c633f5282af842da2d9a4df4 Jun 02 15:47:50.798269 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:50.798180 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g4xl" event={"ID":"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc","Type":"ContainerStarted","Data":"96e1825ef9b3ed58235fb4b4d8c1af5b76b9a4f4c633f5282af842da2d9a4df4"} Jun 02 15:47:51.802644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:51.802555 2578 generic.go:358] "Generic (PLEG): container finished" podID="a2059bfa-367d-4a5c-9d2e-8662fa11a3cc" containerID="fcbb9c19fa6f99ba3652a9b80aef12266cd5bed6c8ef13539946d590cc67aea5" exitCode=0 Jun 02 15:47:51.802644 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:51.802625 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g4xl" event={"ID":"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc","Type":"ContainerDied","Data":"fcbb9c19fa6f99ba3652a9b80aef12266cd5bed6c8ef13539946d590cc67aea5"} Jun 02 15:47:52.807967 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:52.807929 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g4xl" event={"ID":"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc","Type":"ContainerStarted","Data":"ddc3c65a5677c815de39248c0f01cefaef40e2841d3f87ea5eb2e994302c4af6"} Jun 02 15:47:52.807967 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:52.807974 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g4xl" event={"ID":"a2059bfa-367d-4a5c-9d2e-8662fa11a3cc","Type":"ContainerStarted","Data":"a52a643ba07c7106436ad2153a245c61d6a620baac434ac4d2f3888d57ff6cb0"} Jun 02 15:47:52.858513 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:52.858464 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7g4xl" podStartSLOduration=2.997183163 podStartE2EDuration="3.858449403s" podCreationTimestamp="2026-06-02 15:47:49 +0000 UTC" firstStartedPulling="2026-06-02 15:47:50.635189609 +0000 UTC m=+81.764525811" lastFinishedPulling="2026-06-02 15:47:51.496455845 +0000 UTC m=+82.625792051" observedRunningTime="2026-06-02 15:47:52.854976226 +0000 UTC m=+83.984312448" watchObservedRunningTime="2026-06-02 15:47:52.858449403 +0000 UTC m=+83.987785646" Jun 02 15:47:55.366818 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:55.366785 2578 patch_prober.go:28] interesting pod/image-registry-84b49587fd-824gx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 15:47:55.367168 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:55.366839 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84b49587fd-824gx" podUID="626b807d-b328-4e35-9d57-48fe8ecc46f4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:47:57.743877 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:47:57.743846 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:48:00.989524 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:00.989487 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84b49587fd-824gx"] Jun 02 15:48:06.734227 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:06.734197 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-94lpx" Jun 02 15:48:20.887174 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:20.887143 2578 generic.go:358] "Generic (PLEG): container finished" podID="87b11978-f71a-49d3-bf1e-2eb41d30143b" containerID="7d06d3e52508699e6d1a1be6e3c3288bc5cc07214e6dad8d44c24b639769cd5a" exitCode=0 Jun 02 15:48:20.887588 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:20.887208 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" event={"ID":"87b11978-f71a-49d3-bf1e-2eb41d30143b","Type":"ContainerDied","Data":"7d06d3e52508699e6d1a1be6e3c3288bc5cc07214e6dad8d44c24b639769cd5a"} Jun 02 15:48:20.887588 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:20.887579 2578 scope.go:117] "RemoveContainer" containerID="7d06d3e52508699e6d1a1be6e3c3288bc5cc07214e6dad8d44c24b639769cd5a" Jun 02 15:48:21.891538 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:21.891494 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-7zqxp" event={"ID":"87b11978-f71a-49d3-bf1e-2eb41d30143b","Type":"ContainerStarted","Data":"f891567953b1efb621d7495e44b5ebf02159dfbe039f0b0f215ccb862909be10"} Jun 02 15:48:25.903868 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:25.903835 2578 generic.go:358] "Generic (PLEG): container finished" podID="0cd57594-0419-401e-8406-38e65a8e0ef5" containerID="77cf63201ef18e2df0f566f4ef168a232409409ccaee0c62ecc4566975e84fb7" exitCode=0 Jun 02 15:48:25.904265 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:25.903908 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" event={"ID":"0cd57594-0419-401e-8406-38e65a8e0ef5","Type":"ContainerDied","Data":"77cf63201ef18e2df0f566f4ef168a232409409ccaee0c62ecc4566975e84fb7"} Jun 02 15:48:25.904265 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:25.904205 2578 scope.go:117] "RemoveContainer" containerID="77cf63201ef18e2df0f566f4ef168a232409409ccaee0c62ecc4566975e84fb7" Jun 02 15:48:26.009755 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.009687 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84b49587fd-824gx" podUID="626b807d-b328-4e35-9d57-48fe8ecc46f4" containerName="registry" containerID="cri-o://b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e" gracePeriod=30 Jun 02 15:48:26.276460 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.276434 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:48:26.381377 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.381335 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") pod \"626b807d-b328-4e35-9d57-48fe8ecc46f4\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " Jun 02 15:48:26.381583 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.381401 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-installation-pull-secrets\") pod \"626b807d-b328-4e35-9d57-48fe8ecc46f4\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " Jun 02 15:48:26.381583 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.381503 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-certificates\") pod \"626b807d-b328-4e35-9d57-48fe8ecc46f4\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " Jun 02 15:48:26.381583 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.381536 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-bound-sa-token\") pod \"626b807d-b328-4e35-9d57-48fe8ecc46f4\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " Jun 02 15:48:26.381583 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.381564 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-trusted-ca\") pod \"626b807d-b328-4e35-9d57-48fe8ecc46f4\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " Jun 02 15:48:26.381785 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.381591 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/626b807d-b328-4e35-9d57-48fe8ecc46f4-ca-trust-extracted\") pod \"626b807d-b328-4e35-9d57-48fe8ecc46f4\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " Jun 02 15:48:26.381785 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.381626 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-image-registry-private-configuration\") pod \"626b807d-b328-4e35-9d57-48fe8ecc46f4\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " Jun 02 15:48:26.381785 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.381669 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdpjw\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-kube-api-access-cdpjw\") pod \"626b807d-b328-4e35-9d57-48fe8ecc46f4\" (UID: \"626b807d-b328-4e35-9d57-48fe8ecc46f4\") " Jun 02 15:48:26.382441 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.382037 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "626b807d-b328-4e35-9d57-48fe8ecc46f4" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:26.382441 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.382361 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "626b807d-b328-4e35-9d57-48fe8ecc46f4" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:26.384313 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.384286 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-kube-api-access-cdpjw" (OuterVolumeSpecName: "kube-api-access-cdpjw") pod "626b807d-b328-4e35-9d57-48fe8ecc46f4" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4"). InnerVolumeSpecName "kube-api-access-cdpjw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:48:26.384485 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.384288 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "626b807d-b328-4e35-9d57-48fe8ecc46f4" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:48:26.384556 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.384503 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "626b807d-b328-4e35-9d57-48fe8ecc46f4" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:48:26.384556 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.384521 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "626b807d-b328-4e35-9d57-48fe8ecc46f4" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:48:26.384658 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.384590 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "626b807d-b328-4e35-9d57-48fe8ecc46f4" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:48:26.390972 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.390945 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626b807d-b328-4e35-9d57-48fe8ecc46f4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "626b807d-b328-4e35-9d57-48fe8ecc46f4" (UID: "626b807d-b328-4e35-9d57-48fe8ecc46f4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 15:48:26.482956 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.482865 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-certificates\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:48:26.482956 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.482896 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-bound-sa-token\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:48:26.482956 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.482907 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/626b807d-b328-4e35-9d57-48fe8ecc46f4-trusted-ca\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:48:26.482956 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.482915 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/626b807d-b328-4e35-9d57-48fe8ecc46f4-ca-trust-extracted\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:48:26.482956 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.482925 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-image-registry-private-configuration\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:48:26.482956 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.482935 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdpjw\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-kube-api-access-cdpjw\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:48:26.482956 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.482943 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/626b807d-b328-4e35-9d57-48fe8ecc46f4-registry-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:48:26.482956 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.482952 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/626b807d-b328-4e35-9d57-48fe8ecc46f4-installation-pull-secrets\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:48:26.910081 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.910042 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2q62h" event={"ID":"0cd57594-0419-401e-8406-38e65a8e0ef5","Type":"ContainerStarted","Data":"180d5b0a36b16f624d6e475b209856feafab28d0276d27c12bf3200eee8ddf96"} Jun 02 15:48:26.911593 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.911563 2578 generic.go:358] "Generic (PLEG): container finished" podID="626b807d-b328-4e35-9d57-48fe8ecc46f4" containerID="b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e" exitCode=0 Jun 02 15:48:26.911745 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.911634 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84b49587fd-824gx" Jun 02 15:48:26.911745 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.911641 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84b49587fd-824gx" event={"ID":"626b807d-b328-4e35-9d57-48fe8ecc46f4","Type":"ContainerDied","Data":"b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e"} Jun 02 15:48:26.911745 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.911667 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84b49587fd-824gx" event={"ID":"626b807d-b328-4e35-9d57-48fe8ecc46f4","Type":"ContainerDied","Data":"60895adf142c3f97f7addb4050fb6ce5829eb9b754688d049a28b8d200235273"} Jun 02 15:48:26.911745 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.911686 2578 scope.go:117] "RemoveContainer" containerID="b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e" Jun 02 15:48:26.925338 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.924396 2578 scope.go:117] "RemoveContainer" containerID="b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e" Jun 02 15:48:26.925647 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:48:26.925621 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e\": container with ID starting with b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e not found: ID does not exist" containerID="b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e" Jun 02 15:48:26.925700 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.925655 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e"} err="failed to get container status \"b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e\": rpc error: code = NotFound desc = could not find container \"b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e\": container with ID starting with b39d564f2043427954bf53c8dded8af00dd36a402323ef9af547f02d1f6e159e not found: ID does not exist" Jun 02 15:48:26.950826 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.950796 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84b49587fd-824gx"] Jun 02 15:48:26.965085 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:26.965047 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84b49587fd-824gx"] Jun 02 15:48:27.414518 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:27.414483 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="626b807d-b328-4e35-9d57-48fe8ecc46f4" path="/var/lib/kubelet/pods/626b807d-b328-4e35-9d57-48fe8ecc46f4/volumes" Jun 02 15:48:30.924923 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:30.924887 2578 generic.go:358] "Generic (PLEG): container finished" podID="b9b9abd4-4ecd-44ca-9329-a1923368bd46" containerID="577f79d4af84da7fee153f2e0086734f1ab91cc688eb19080bf6405edd182b6d" exitCode=0 Jun 02 15:48:30.925353 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:30.924960 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-chj4l" event={"ID":"b9b9abd4-4ecd-44ca-9329-a1923368bd46","Type":"ContainerDied","Data":"577f79d4af84da7fee153f2e0086734f1ab91cc688eb19080bf6405edd182b6d"} Jun 02 15:48:30.925353 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:30.925302 2578 scope.go:117] "RemoveContainer" containerID="577f79d4af84da7fee153f2e0086734f1ab91cc688eb19080bf6405edd182b6d" Jun 02 15:48:31.931698 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:31.931665 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-chj4l" event={"ID":"b9b9abd4-4ecd-44ca-9329-a1923368bd46","Type":"ContainerStarted","Data":"fcbe2942659c534e345d96a776f7bca40f33410c14545976ac1c912784f20723"} Jun 02 15:48:52.230569 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.230532 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z"] Jun 02 15:48:52.231180 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.230965 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626b807d-b328-4e35-9d57-48fe8ecc46f4" containerName="registry" Jun 02 15:48:52.231180 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.230983 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="626b807d-b328-4e35-9d57-48fe8ecc46f4" containerName="registry" Jun 02 15:48:52.231180 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.231045 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="626b807d-b328-4e35-9d57-48fe8ecc46f4" containerName="registry" Jun 02 15:48:52.235191 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.235170 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.237804 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.237782 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 02 15:48:52.237909 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.237823 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 02 15:48:52.240557 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.240537 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 02 15:48:52.240850 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.240835 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 02 15:48:52.245767 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.245748 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z"] Jun 02 15:48:52.290238 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.290208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/337dc52a-cab0-4e8a-b1f6-a46bd141222a-klusterlet-config\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.290424 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.290262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/337dc52a-cab0-4e8a-b1f6-a46bd141222a-tmp\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.290424 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.290310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjk27\" (UniqueName: \"kubernetes.io/projected/337dc52a-cab0-4e8a-b1f6-a46bd141222a-kube-api-access-gjk27\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.391834 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.391754 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/337dc52a-cab0-4e8a-b1f6-a46bd141222a-tmp\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.391834 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.391815 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjk27\" (UniqueName: \"kubernetes.io/projected/337dc52a-cab0-4e8a-b1f6-a46bd141222a-kube-api-access-gjk27\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.392038 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.391867 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/337dc52a-cab0-4e8a-b1f6-a46bd141222a-klusterlet-config\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.392197 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.392177 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/337dc52a-cab0-4e8a-b1f6-a46bd141222a-tmp\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.394309 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.394280 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/337dc52a-cab0-4e8a-b1f6-a46bd141222a-klusterlet-config\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.403902 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.403879 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2"] Jun 02 15:48:52.405102 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.405071 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjk27\" (UniqueName: \"kubernetes.io/projected/337dc52a-cab0-4e8a-b1f6-a46bd141222a-kube-api-access-gjk27\") pod \"klusterlet-addon-workmgr-d95f4f47d-mnj8z\" (UID: \"337dc52a-cab0-4e8a-b1f6-a46bd141222a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.407329 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.407312 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.410084 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.410051 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 02 15:48:52.410182 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.410098 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 02 15:48:52.410182 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.410105 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 02 15:48:52.410885 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.410869 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 02 15:48:52.421650 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.421626 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2"] Jun 02 15:48:52.492699 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.492603 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-hub\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.492699 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.492641 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.492699 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.492661 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-ca\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.492699 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.492684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/4054459a-df33-44c0-a32c-c6e3740cf9a5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.492954 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.492760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.492954 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.492806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7b2l\" (UniqueName: \"kubernetes.io/projected/4054459a-df33-44c0-a32c-c6e3740cf9a5-kube-api-access-k7b2l\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.544657 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.544620 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:52.593834 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.593803 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-hub\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.593966 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.593838 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.593966 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.593860 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-ca\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.593966 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.593966 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/4054459a-df33-44c0-a32c-c6e3740cf9a5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.594160 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.593992 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.594160 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.594011 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k7b2l\" (UniqueName: \"kubernetes.io/projected/4054459a-df33-44c0-a32c-c6e3740cf9a5-kube-api-access-k7b2l\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.594940 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.594912 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/4054459a-df33-44c0-a32c-c6e3740cf9a5-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.596459 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.596423 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-ca\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.596944 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.596907 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.597103 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.597078 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-hub\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.597510 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.597488 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4054459a-df33-44c0-a32c-c6e3740cf9a5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.604343 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.604287 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7b2l\" (UniqueName: \"kubernetes.io/projected/4054459a-df33-44c0-a32c-c6e3740cf9a5-kube-api-access-k7b2l\") pod \"cluster-proxy-proxy-agent-78d99767fc-8pgd2\" (UID: \"4054459a-df33-44c0-a32c-c6e3740cf9a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.671759 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.671660 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z"] Jun 02 15:48:52.674301 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:48:52.674277 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337dc52a_cab0_4e8a_b1f6_a46bd141222a.slice/crio-a862f6a4c58e3ff4dd2ecd6d8f07d4f812fc85ebe68d22a99c0e2c6eb1240cc9 WatchSource:0}: Error finding container a862f6a4c58e3ff4dd2ecd6d8f07d4f812fc85ebe68d22a99c0e2c6eb1240cc9: Status 404 returned error can't find the container with id a862f6a4c58e3ff4dd2ecd6d8f07d4f812fc85ebe68d22a99c0e2c6eb1240cc9 Jun 02 15:48:52.733690 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.733661 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" Jun 02 15:48:52.871299 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.871141 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2"] Jun 02 15:48:52.873668 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:48:52.873644 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4054459a_df33_44c0_a32c_c6e3740cf9a5.slice/crio-b1fbc4f22ae0ec4510299d7b5074184611b12b14b29ac58226de2ee63b124e2f WatchSource:0}: Error finding container b1fbc4f22ae0ec4510299d7b5074184611b12b14b29ac58226de2ee63b124e2f: Status 404 returned error can't find the container with id b1fbc4f22ae0ec4510299d7b5074184611b12b14b29ac58226de2ee63b124e2f Jun 02 15:48:52.993935 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.993897 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" event={"ID":"337dc52a-cab0-4e8a-b1f6-a46bd141222a","Type":"ContainerStarted","Data":"a862f6a4c58e3ff4dd2ecd6d8f07d4f812fc85ebe68d22a99c0e2c6eb1240cc9"} Jun 02 15:48:52.994916 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:52.994883 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" event={"ID":"4054459a-df33-44c0-a32c-c6e3740cf9a5","Type":"ContainerStarted","Data":"b1fbc4f22ae0ec4510299d7b5074184611b12b14b29ac58226de2ee63b124e2f"} Jun 02 15:48:58.016270 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:58.016157 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" event={"ID":"337dc52a-cab0-4e8a-b1f6-a46bd141222a","Type":"ContainerStarted","Data":"17a2c4e7988c63ce1b02c614b86354491368884777df79f33076c8d16d9434fd"} Jun 02 15:48:58.016743 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:58.016327 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:58.017497 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:58.017467 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" event={"ID":"4054459a-df33-44c0-a32c-c6e3740cf9a5","Type":"ContainerStarted","Data":"48da8eeaa46e649a2df8b9ee586508bafa92c2ea466e253a510219e26f3785c6"} Jun 02 15:48:58.018250 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:58.018233 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" Jun 02 15:48:58.035938 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:48:58.035887 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d95f4f47d-mnj8z" podStartSLOduration=0.954080839 podStartE2EDuration="6.03586921s" podCreationTimestamp="2026-06-02 15:48:52 +0000 UTC" firstStartedPulling="2026-06-02 15:48:52.676118083 +0000 UTC m=+143.805454284" lastFinishedPulling="2026-06-02 15:48:57.757906451 +0000 UTC m=+148.887242655" observedRunningTime="2026-06-02 15:48:58.034465078 +0000 UTC m=+149.163801301" watchObservedRunningTime="2026-06-02 15:48:58.03586921 +0000 UTC m=+149.165205490" Jun 02 15:49:01.027836 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:49:01.027795 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" event={"ID":"4054459a-df33-44c0-a32c-c6e3740cf9a5","Type":"ContainerStarted","Data":"e91e2b9d9afdc735918811f2470c8d7ee849213a891bec194e9fedc491264bb3"} Jun 02 15:49:01.027836 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:49:01.027839 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" event={"ID":"4054459a-df33-44c0-a32c-c6e3740cf9a5","Type":"ContainerStarted","Data":"1a6c7afcb45035bbbe7820873327626c6e72de3de76261f3d6450e0ead574ec2"} Jun 02 15:49:01.051978 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:49:01.051921 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d99767fc-8pgd2" podStartSLOduration=1.758992749 podStartE2EDuration="9.051904177s" podCreationTimestamp="2026-06-02 15:48:52 +0000 UTC" firstStartedPulling="2026-06-02 15:48:52.875483539 +0000 UTC m=+144.004819739" lastFinishedPulling="2026-06-02 15:49:00.168394962 +0000 UTC m=+151.297731167" observedRunningTime="2026-06-02 15:49:01.050793836 +0000 UTC m=+152.180130059" watchObservedRunningTime="2026-06-02 15:49:01.051904177 +0000 UTC m=+152.181240468" Jun 02 15:51:29.326460 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:51:29.326426 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 15:51:29.327058 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:51:29.327037 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 15:51:29.336251 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:51:29.336228 2578 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 15:55:01.143050 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.143010 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-86f7b4b499-zbvvd"] Jun 02 15:55:01.146041 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.146022 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:01.148143 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.148123 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jun 02 15:55:01.148265 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.148150 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jun 02 15:55:01.148265 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.148155 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jun 02 15:55:01.148559 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.148540 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-4wm64\"" Jun 02 15:55:01.159087 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.159059 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-zbvvd"] Jun 02 15:55:01.222820 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.222780 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp266\" (UniqueName: \"kubernetes.io/projected/8fe54e74-a7e1-4c52-9e21-793740d16261-kube-api-access-cp266\") pod \"model-serving-api-86f7b4b499-zbvvd\" (UID: \"8fe54e74-a7e1-4c52-9e21-793740d16261\") " pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:01.222820 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.222821 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe54e74-a7e1-4c52-9e21-793740d16261-tls-certs\") pod \"model-serving-api-86f7b4b499-zbvvd\" (UID: \"8fe54e74-a7e1-4c52-9e21-793740d16261\") " pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:01.323719 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.323682 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cp266\" (UniqueName: \"kubernetes.io/projected/8fe54e74-a7e1-4c52-9e21-793740d16261-kube-api-access-cp266\") pod \"model-serving-api-86f7b4b499-zbvvd\" (UID: \"8fe54e74-a7e1-4c52-9e21-793740d16261\") " pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:01.323719 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.323723 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe54e74-a7e1-4c52-9e21-793740d16261-tls-certs\") pod \"model-serving-api-86f7b4b499-zbvvd\" (UID: \"8fe54e74-a7e1-4c52-9e21-793740d16261\") " pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:01.323928 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:55:01.323835 2578 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 02 15:55:01.323928 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:55:01.323899 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe54e74-a7e1-4c52-9e21-793740d16261-tls-certs podName:8fe54e74-a7e1-4c52-9e21-793740d16261 nodeName:}" failed. No retries permitted until 2026-06-02 15:55:01.823879804 +0000 UTC m=+512.953216013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8fe54e74-a7e1-4c52-9e21-793740d16261-tls-certs") pod "model-serving-api-86f7b4b499-zbvvd" (UID: "8fe54e74-a7e1-4c52-9e21-793740d16261") : secret "model-serving-api-tls" not found Jun 02 15:55:01.335845 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.335819 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp266\" (UniqueName: \"kubernetes.io/projected/8fe54e74-a7e1-4c52-9e21-793740d16261-kube-api-access-cp266\") pod \"model-serving-api-86f7b4b499-zbvvd\" (UID: \"8fe54e74-a7e1-4c52-9e21-793740d16261\") " pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:01.828622 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.828583 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe54e74-a7e1-4c52-9e21-793740d16261-tls-certs\") pod \"model-serving-api-86f7b4b499-zbvvd\" (UID: \"8fe54e74-a7e1-4c52-9e21-793740d16261\") " pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:01.831124 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:01.831097 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe54e74-a7e1-4c52-9e21-793740d16261-tls-certs\") pod \"model-serving-api-86f7b4b499-zbvvd\" (UID: \"8fe54e74-a7e1-4c52-9e21-793740d16261\") " pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:02.056378 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:02.056333 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:02.186394 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:02.186341 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-zbvvd"] Jun 02 15:55:02.189293 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:55:02.189265 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe54e74_a7e1_4c52_9e21_793740d16261.slice/crio-b4a60d81ac6bb4104a9bfe0dd21c0d73387a7386db8f4c45b7fde38d7ef07cdb WatchSource:0}: Error finding container b4a60d81ac6bb4104a9bfe0dd21c0d73387a7386db8f4c45b7fde38d7ef07cdb: Status 404 returned error can't find the container with id b4a60d81ac6bb4104a9bfe0dd21c0d73387a7386db8f4c45b7fde38d7ef07cdb Jun 02 15:55:02.191441 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:02.191423 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 15:55:03.011472 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:03.011434 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-zbvvd" event={"ID":"8fe54e74-a7e1-4c52-9e21-793740d16261","Type":"ContainerStarted","Data":"b4a60d81ac6bb4104a9bfe0dd21c0d73387a7386db8f4c45b7fde38d7ef07cdb"} Jun 02 15:55:05.018942 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:05.018906 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-zbvvd" event={"ID":"8fe54e74-a7e1-4c52-9e21-793740d16261","Type":"ContainerStarted","Data":"93d36da92f825d1a7ab4eef84d2719981a6993c01ab95f59befe09ebe7f0b4a2"} Jun 02 15:55:05.019344 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:05.018966 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:05.040712 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:05.040657 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/model-serving-api-86f7b4b499-zbvvd" podStartSLOduration=1.743034637 podStartE2EDuration="4.040641945s" podCreationTimestamp="2026-06-02 15:55:01 +0000 UTC" firstStartedPulling="2026-06-02 15:55:02.191582507 +0000 UTC m=+513.320918708" lastFinishedPulling="2026-06-02 15:55:04.48918981 +0000 UTC m=+515.618526016" observedRunningTime="2026-06-02 15:55:05.039099219 +0000 UTC m=+516.168435441" watchObservedRunningTime="2026-06-02 15:55:05.040641945 +0000 UTC m=+516.169978167" Jun 02 15:55:16.026229 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:16.026199 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-86f7b4b499-zbvvd" Jun 02 15:55:37.321617 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.321583 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2"] Jun 02 15:55:37.325979 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.325955 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.328420 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.328396 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jun 02 15:55:37.328844 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.328822 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jun 02 15:55:37.328942 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.328883 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jun 02 15:55:37.328942 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.328918 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g84l2\"" Jun 02 15:55:37.328942 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.328930 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jun 02 15:55:37.336388 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.336349 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2"] Jun 02 15:55:37.422021 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.421984 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.422021 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.422032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv6w2\" (UniqueName: \"kubernetes.io/projected/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kube-api-access-rv6w2\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.422277 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.422128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.422277 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.422204 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.522731 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.522688 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.522731 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.522736 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rv6w2\" (UniqueName: \"kubernetes.io/projected/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kube-api-access-rv6w2\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.522991 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.522770 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.522991 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.522809 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.523261 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.523239 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.523514 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.523495 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.525304 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.525276 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.532583 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.532558 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv6w2\" (UniqueName: \"kubernetes.io/projected/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kube-api-access-rv6w2\") pod \"isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.636714 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.636618 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:55:37.770942 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:37.770907 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2"] Jun 02 15:55:37.774154 ip-10-0-128-123 kubenswrapper[2578]: W0602 15:55:37.774126 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5ce4f3_fa1f_407c_b1a5_42f55604a2ca.slice/crio-1ed688ee23a9e1a73608462f53a901c46ee7fe31966bb74341ba64e0e5a7dc25 WatchSource:0}: Error finding container 1ed688ee23a9e1a73608462f53a901c46ee7fe31966bb74341ba64e0e5a7dc25: Status 404 returned error can't find the container with id 1ed688ee23a9e1a73608462f53a901c46ee7fe31966bb74341ba64e0e5a7dc25 Jun 02 15:55:38.112765 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:38.112708 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" event={"ID":"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca","Type":"ContainerStarted","Data":"1ed688ee23a9e1a73608462f53a901c46ee7fe31966bb74341ba64e0e5a7dc25"} Jun 02 15:55:41.125024 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:41.124939 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" event={"ID":"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca","Type":"ContainerStarted","Data":"9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844"} Jun 02 15:55:45.138643 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:45.138605 2578 generic.go:358] "Generic (PLEG): container finished" podID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerID="9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844" exitCode=0 Jun 02 15:55:45.139085 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:45.138677 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" event={"ID":"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca","Type":"ContainerDied","Data":"9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844"} Jun 02 15:55:59.190077 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:55:59.190040 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" event={"ID":"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca","Type":"ContainerStarted","Data":"eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074"} Jun 02 15:56:01.198967 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:01.198931 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" event={"ID":"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca","Type":"ContainerStarted","Data":"752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878"} Jun 02 15:56:01.199344 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:01.199127 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:56:01.225271 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:01.225210 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podStartSLOduration=1.526182714 podStartE2EDuration="24.225193349s" podCreationTimestamp="2026-06-02 15:55:37 +0000 UTC" firstStartedPulling="2026-06-02 15:55:37.776055651 +0000 UTC m=+548.905391852" lastFinishedPulling="2026-06-02 15:56:00.475066281 +0000 UTC m=+571.604402487" observedRunningTime="2026-06-02 15:56:01.223216665 +0000 UTC m=+572.352552887" watchObservedRunningTime="2026-06-02 15:56:01.225193349 +0000 UTC m=+572.354529572" Jun 02 15:56:02.201587 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:02.201555 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:56:02.202897 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:02.202867 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:56:03.204383 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:03.204320 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:56:08.209415 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:08.209360 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:56:08.209989 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:08.209954 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:56:18.210597 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:18.210556 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:56:28.209841 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:28.209798 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:56:29.349862 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:29.349834 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 15:56:29.350293 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:29.350084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 15:56:38.210782 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:38.210741 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:56:48.210324 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:48.210226 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:56:58.210788 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:56:58.210751 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:57:08.211268 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:57:08.211230 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:57:56.526015 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:57:56.525976 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2"] Jun 02 15:57:56.526661 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:57:56.526304 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" containerID="cri-o://eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074" gracePeriod=30 Jun 02 15:57:56.526661 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:57:56.526330 2578 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kube-rbac-proxy" containerID="cri-o://752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878" gracePeriod=30 Jun 02 15:57:57.528518 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:57:57.528484 2578 generic.go:358] "Generic (PLEG): container finished" podID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerID="752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878" exitCode=2 Jun 02 15:57:57.528901 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:57:57.528556 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" event={"ID":"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca","Type":"ContainerDied","Data":"752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878"} Jun 02 15:57:58.205474 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:57:58.205434 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.23:8643/healthz\": dial tcp 10.132.0.23:8643: connect: connection refused" Jun 02 15:57:58.209853 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:57:58.209829 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused" Jun 02 15:58:00.962254 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:00.962223 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:58:01.149237 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.149204 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-proxy-tls\") pod \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " Jun 02 15:58:01.149237 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.149241 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kserve-provision-location\") pod \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " Jun 02 15:58:01.149475 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.149336 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv6w2\" (UniqueName: \"kubernetes.io/projected/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kube-api-access-rv6w2\") pod \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " Jun 02 15:58:01.149475 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.149375 2578 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\") pod \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\" (UID: \"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca\") " Jun 02 15:58:01.149726 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.149697 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" (UID: "8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 15:58:01.149816 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.149755 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-isvc-sklearn-graph-2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "isvc-sklearn-graph-2-kube-rbac-proxy-sar-config") pod "8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" (UID: "8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca"). InnerVolumeSpecName "isvc-sklearn-graph-2-kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:58:01.151531 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.151497 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kube-api-access-rv6w2" (OuterVolumeSpecName: "kube-api-access-rv6w2") pod "8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" (UID: "8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca"). InnerVolumeSpecName "kube-api-access-rv6w2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:58:01.151639 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.151564 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" (UID: "8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:58:01.250539 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.250498 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv6w2\" (UniqueName: \"kubernetes.io/projected/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kube-api-access-rv6w2\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:58:01.250539 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.250538 2578 reconciler_common.go:299] "Volume detached for volume \"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:58:01.250741 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.250551 2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-proxy-tls\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:58:01.250741 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.250561 2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca-kserve-provision-location\") on node \"ip-10-0-128-123.ec2.internal\" DevicePath \"\"" Jun 02 15:58:01.540857 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.540822 2578 generic.go:358] "Generic (PLEG): container finished" podID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerID="eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074" exitCode=0 Jun 02 15:58:01.540997 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.540900 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" event={"ID":"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca","Type":"ContainerDied","Data":"eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074"} Jun 02 15:58:01.540997 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.540928 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" event={"ID":"8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca","Type":"ContainerDied","Data":"1ed688ee23a9e1a73608462f53a901c46ee7fe31966bb74341ba64e0e5a7dc25"} Jun 02 15:58:01.540997 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.540935 2578 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2" Jun 02 15:58:01.540997 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.540943 2578 scope.go:117] "RemoveContainer" containerID="752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878" Jun 02 15:58:01.548846 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.548829 2578 scope.go:117] "RemoveContainer" containerID="eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074" Jun 02 15:58:01.555784 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.555764 2578 scope.go:117] "RemoveContainer" containerID="9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844" Jun 02 15:58:01.561512 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.561486 2578 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2"] Jun 02 15:58:01.563037 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.563021 2578 scope.go:117] "RemoveContainer" containerID="752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878" Jun 02 15:58:01.563289 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:58:01.563273 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878\": container with ID starting with 752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878 not found: ID does not exist" containerID="752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878" Jun 02 15:58:01.563346 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.563297 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878"} err="failed to get container status \"752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878\": rpc error: code = NotFound desc = could not find container \"752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878\": container with ID starting with 752a58900196f251f83e72179c038e60d81e8f2e2dce16c2b2ae4d30c46f6878 not found: ID does not exist" Jun 02 15:58:01.563346 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.563319 2578 scope.go:117] "RemoveContainer" containerID="eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074" Jun 02 15:58:01.563671 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:58:01.563652 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074\": container with ID starting with eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074 not found: ID does not exist" containerID="eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074" Jun 02 15:58:01.563737 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.563681 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074"} err="failed to get container status \"eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074\": rpc error: code = NotFound desc = could not find container \"eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074\": container with ID starting with eeec7fb229d975f195d54b8963ab7027d017523f9cc4c60f1ab1f21852d25074 not found: ID does not exist" Jun 02 15:58:01.563737 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.563704 2578 scope.go:117] "RemoveContainer" containerID="9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844" Jun 02 15:58:01.563947 ip-10-0-128-123 kubenswrapper[2578]: E0602 15:58:01.563923 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844\": container with ID starting with 9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844 not found: ID does not exist" containerID="9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844" Jun 02 15:58:01.563989 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.563946 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844"} err="failed to get container status \"9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844\": rpc error: code = NotFound desc = could not find container \"9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844\": container with ID starting with 9576bbc48400b2f428ea6d288258c7efda78ede7f9c10ec727d84861e292f844 not found: ID does not exist" Jun 02 15:58:01.570200 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:01.570178 2578 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-5d696f88d7-2nwz2"] Jun 02 15:58:03.414746 ip-10-0-128-123 kubenswrapper[2578]: I0602 15:58:03.414714 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" path="/var/lib/kubelet/pods/8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca/volumes" Jun 02 16:01:29.374193 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:01:29.374109 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:01:29.375138 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:01:29.375113 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:06:29.394282 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:06:29.394252 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:06:29.396040 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:06:29.396020 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:11:29.417242 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:11:29.417212 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:11:29.420327 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:11:29.420306 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:16:29.439577 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:16:29.439546 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:16:29.443271 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:16:29.443248 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:21:29.459701 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:21:29.459672 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:21:29.464724 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:21:29.464696 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:26:29.479445 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:26:29.479315 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:26:29.489930 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:26:29.489908 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:31:29.499325 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:31:29.499224 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:31:29.508802 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:31:29.508781 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:35:30.766496 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:30.766460 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5f5cp_d8949689-7df4-4ab2-b219-e6bc3ee79e40/global-pull-secret-syncer/0.log" Jun 02 16:35:30.994490 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:30.994451 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lvvg8_6f2fd357-da07-4b2a-a98b-29ea324b4971/konnectivity-agent/0.log" Jun 02 16:35:31.015487 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:31.015434 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-123.ec2.internal_131ee351bb55ad6be954235ac6b9f987/haproxy/0.log" Jun 02 16:35:34.712632 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:34.712551 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-69b6cbdb88-s9svn_737dbd8f-f1e2-4152-90a0-c8b86b42a170/cluster-monitoring-operator/0.log" Jun 02 16:35:35.026761 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:35.026678 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g4xl_a2059bfa-367d-4a5c-9d2e-8662fa11a3cc/node-exporter/0.log" Jun 02 16:35:35.053254 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:35.053221 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g4xl_a2059bfa-367d-4a5c-9d2e-8662fa11a3cc/kube-rbac-proxy/0.log" Jun 02 16:35:35.078301 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:35.078273 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g4xl_a2059bfa-367d-4a5c-9d2e-8662fa11a3cc/init-textfile/0.log" Jun 02 16:35:36.933498 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:36.933464 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-hp7hn_efe6293b-b3ab-49e5-ada2-61884c0bed13/networking-console-plugin/0.log" Jun 02 16:35:37.401905 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:37.401867 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/1.log" Jun 02 16:35:37.406537 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:37.406510 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-khtbj_e3ff6dbe-9cfd-4e96-9b58-cc8c07878974/console-operator/2.log" Jun 02 16:35:38.265392 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.265346 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7598fd5bd-77fmb_ed7580d5-42f8-4a02-aa56-1f5b15dff6b1/volume-data-source-validator/0.log" Jun 02 16:35:38.629736 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.629699 2578 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l"] Jun 02 16:35:38.629984 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.629972 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kube-rbac-proxy" Jun 02 16:35:38.630027 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.629986 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kube-rbac-proxy" Jun 02 16:35:38.630027 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.630001 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="storage-initializer" Jun 02 16:35:38.630027 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.630007 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="storage-initializer" Jun 02 16:35:38.630027 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.630019 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" Jun 02 16:35:38.630027 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.630025 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" Jun 02 16:35:38.630177 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.630072 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kube-rbac-proxy" Jun 02 16:35:38.630177 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.630081 2578 memory_manager.go:356] "RemoveStaleState removing state" podUID="8c5ce4f3-fa1f-407c-b1a5-42f55604a2ca" containerName="kserve-container" Jun 02 16:35:38.632986 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.632961 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.634962 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.634944 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pm69g\"/\"openshift-service-ca.crt\"" Jun 02 16:35:38.635408 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.635385 2578 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pm69g\"/\"kube-root-ca.crt\"" Jun 02 16:35:38.635471 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.635385 2578 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pm69g\"/\"default-dockercfg-sjn2p\"" Jun 02 16:35:38.643203 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.643176 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l"] Jun 02 16:35:38.824952 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.824913 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9rzg\" (UniqueName: \"kubernetes.io/projected/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-kube-api-access-l9rzg\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.825130 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.824991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-podres\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.825130 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.825030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-lib-modules\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.825130 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.825055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-sys\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.825130 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.825088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-proc\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.925973 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.925876 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-sys\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.925973 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.925922 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-proc\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.925973 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.925948 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l9rzg\" (UniqueName: \"kubernetes.io/projected/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-kube-api-access-l9rzg\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.926220 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.926013 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-podres\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.926220 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.926018 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-proc\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.926220 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.926024 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-sys\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.926220 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.926082 2578 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-lib-modules\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.926220 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.926146 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-podres\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.926220 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.926185 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-lib-modules\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.935785 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.935754 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9rzg\" (UniqueName: \"kubernetes.io/projected/6c21ec90-2f11-45a4-9f24-9ed61f01b28f-kube-api-access-l9rzg\") pod \"perf-node-gather-daemonset-t8n7l\" (UID: \"6c21ec90-2f11-45a4-9f24-9ed61f01b28f\") " pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:38.942654 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:38.942630 2578 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:39.043532 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.043506 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5jxnm_dd41e666-70e5-4b55-b2d2-bcb8e142678e/dns/0.log" Jun 02 16:35:39.070296 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.070270 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5jxnm_dd41e666-70e5-4b55-b2d2-bcb8e142678e/kube-rbac-proxy/0.log" Jun 02 16:35:39.085943 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.085833 2578 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l"] Jun 02 16:35:39.088751 ip-10-0-128-123 kubenswrapper[2578]: W0602 16:35:39.088718 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6c21ec90_2f11_45a4_9f24_9ed61f01b28f.slice/crio-816c5e13a137e74c9b5e10169724cdec839d2c4343fc651b72e6c9aedbabec3f WatchSource:0}: Error finding container 816c5e13a137e74c9b5e10169724cdec839d2c4343fc651b72e6c9aedbabec3f: Status 404 returned error can't find the container with id 816c5e13a137e74c9b5e10169724cdec839d2c4343fc651b72e6c9aedbabec3f Jun 02 16:35:39.090204 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.090187 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 16:35:39.235130 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.235092 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q4c6g_2de345a5-85e0-422e-8497-e9d7c2fbdcc4/dns-node-resolver/0.log" Jun 02 16:35:39.783728 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.783699 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5ptj6_d794b47c-d098-448e-926e-1e81d554e31f/node-ca/0.log" Jun 02 16:35:39.905911 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.905872 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" event={"ID":"6c21ec90-2f11-45a4-9f24-9ed61f01b28f","Type":"ContainerStarted","Data":"85c3fd3bd9c8e562d9d0fecb1d0d7d906fcb8c917784859aaa829fc77baae1be"} Jun 02 16:35:39.905911 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.905909 2578 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" event={"ID":"6c21ec90-2f11-45a4-9f24-9ed61f01b28f","Type":"ContainerStarted","Data":"816c5e13a137e74c9b5e10169724cdec839d2c4343fc651b72e6c9aedbabec3f"} Jun 02 16:35:39.906106 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.905936 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:39.922028 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:39.921928 2578 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" podStartSLOduration=1.92191138 podStartE2EDuration="1.92191138s" podCreationTimestamp="2026-06-02 16:35:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:35:39.921179792 +0000 UTC m=+2951.050516015" watchObservedRunningTime="2026-06-02 16:35:39.92191138 +0000 UTC m=+2951.051247603" Jun 02 16:35:40.625131 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:40.625102 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6985979865-kdkr9_2b93a739-0f4e-4492-9616-9f2ae397ecd0/router/0.log" Jun 02 16:35:41.023858 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:41.023776 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r8wp6_6dd709d8-d6a3-42aa-bef6-9fa70025b49b/serve-healthcheck-canary/0.log" Jun 02 16:35:41.441190 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:41.441161 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-chj4l_b9b9abd4-4ecd-44ca-9329-a1923368bd46/insights-operator/0.log" Jun 02 16:35:41.441390 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:41.441238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-chj4l_b9b9abd4-4ecd-44ca-9329-a1923368bd46/insights-operator/1.log" Jun 02 16:35:41.542497 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:41.542472 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8ftl_141ed18e-35e0-408a-aaeb-e68b1c5c4e21/kube-rbac-proxy/0.log" Jun 02 16:35:41.566101 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:41.566071 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8ftl_141ed18e-35e0-408a-aaeb-e68b1c5c4e21/exporter/0.log" Jun 02 16:35:41.593037 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:41.593002 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8ftl_141ed18e-35e0-408a-aaeb-e68b1c5c4e21/extractor/0.log" Jun 02 16:35:43.742930 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:43.742902 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-86f7b4b499-zbvvd_8fe54e74-a7e1-4c52-9e21-793740d16261/server/0.log" Jun 02 16:35:45.918858 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:45.918828 2578 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pm69g/perf-node-gather-daemonset-t8n7l" Jun 02 16:35:48.594170 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:48.594138 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-2q62h_0cd57594-0419-401e-8406-38e65a8e0ef5/kube-storage-version-migrator-operator/1.log" Jun 02 16:35:48.595114 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:48.595098 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-2q62h_0cd57594-0419-401e-8406-38e65a8e0ef5/kube-storage-version-migrator-operator/0.log" Jun 02 16:35:49.838092 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:49.838066 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qlzs5_e430bfcf-7490-4927-b6cc-f89f9d081f39/kube-multus-additional-cni-plugins/0.log" Jun 02 16:35:49.871536 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:49.871508 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qlzs5_e430bfcf-7490-4927-b6cc-f89f9d081f39/egress-router-binary-copy/0.log" Jun 02 16:35:49.897306 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:49.897282 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qlzs5_e430bfcf-7490-4927-b6cc-f89f9d081f39/cni-plugins/0.log" Jun 02 16:35:49.923857 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:49.923819 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qlzs5_e430bfcf-7490-4927-b6cc-f89f9d081f39/bond-cni-plugin/0.log" Jun 02 16:35:49.949033 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:49.949009 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qlzs5_e430bfcf-7490-4927-b6cc-f89f9d081f39/routeoverride-cni/0.log" Jun 02 16:35:49.974143 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:49.974116 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qlzs5_e430bfcf-7490-4927-b6cc-f89f9d081f39/whereabouts-cni-bincopy/0.log" Jun 02 16:35:50.003298 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:50.003270 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qlzs5_e430bfcf-7490-4927-b6cc-f89f9d081f39/whereabouts-cni/0.log" Jun 02 16:35:50.321076 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:50.321046 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zc6zt_7a43ecc6-b5eb-435b-9737-ee046a1013ab/kube-multus/0.log" Jun 02 16:35:50.346127 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:50.346095 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7wvwq_877796b1-2b26-4c87-903a-1235f441e838/network-metrics-daemon/0.log" Jun 02 16:35:50.369858 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:50.369833 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7wvwq_877796b1-2b26-4c87-903a-1235f441e838/kube-rbac-proxy/0.log" Jun 02 16:35:51.312915 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:51.312884 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnkrg_6d0b6596-6b7d-4228-85d3-95620e2804a7/ovn-controller/0.log" Jun 02 16:35:51.348258 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:51.348231 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnkrg_6d0b6596-6b7d-4228-85d3-95620e2804a7/ovn-acl-logging/0.log" Jun 02 16:35:51.368385 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:51.368336 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnkrg_6d0b6596-6b7d-4228-85d3-95620e2804a7/kube-rbac-proxy-node/0.log" Jun 02 16:35:51.395157 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:51.395129 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnkrg_6d0b6596-6b7d-4228-85d3-95620e2804a7/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 16:35:51.418779 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:51.418755 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnkrg_6d0b6596-6b7d-4228-85d3-95620e2804a7/northd/0.log" Jun 02 16:35:51.443797 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:51.443768 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnkrg_6d0b6596-6b7d-4228-85d3-95620e2804a7/nbdb/0.log" Jun 02 16:35:51.468214 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:51.468182 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnkrg_6d0b6596-6b7d-4228-85d3-95620e2804a7/sbdb/0.log" Jun 02 16:35:51.564960 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:51.564883 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnkrg_6d0b6596-6b7d-4228-85d3-95620e2804a7/ovnkube-controller/0.log" Jun 02 16:35:53.563909 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:53.563881 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56d84b9f44-98rxb_46349232-ad6b-4e6d-851e-53efe1621539/check-endpoints/0.log" Jun 02 16:35:53.593104 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:53.593075 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-94lpx_3a94647c-a5b5-4c9d-a13d-d6c824a3a79e/network-check-target-container/0.log" Jun 02 16:35:54.688229 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:54.688201 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-wpk7x_010e3146-6f24-4c4f-853e-899ae1a2fe39/iptables-alerter/0.log" Jun 02 16:35:55.493133 ip-10-0-128-123 kubenswrapper[2578]: I0602 16:35:55.493101 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-thdlh_893b281a-ffc7-4fdd-b162-738f9f32da0c/tuned/0.log"