Jun 07 09:08:07.737180 ip-10-0-129-203 systemd[1]: Starting Kubernetes Kubelet... Jun 07 09:08:08.140090 ip-10-0-129-203 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 07 09:08:08.140090 ip-10-0-129-203 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 07 09:08:08.140090 ip-10-0-129-203 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 07 09:08:08.140090 ip-10-0-129-203 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 07 09:08:08.140090 ip-10-0-129-203 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 07 09:08:08.142284 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.142198 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 07 09:08:08.148102 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148087 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:08.148102 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148101 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148105 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148108 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148113 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148117 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148120 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148123 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148126 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148129 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148132 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148135 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148138 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148140 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148143 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148146 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148148 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148152 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148154 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148162 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:08.148172 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148165 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148168 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148171 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148173 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148176 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148180 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148183 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148187 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148189 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148192 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148194 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148197 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148199 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148203 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148207 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148210 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148213 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148216 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148219 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148222 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:08.148651 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148225 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148228 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148230 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148233 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148235 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148238 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148240 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148243 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148245 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148248 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148250 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148253 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148255 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148257 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148262 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148264 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148267 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148270 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148272 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148275 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:08.149560 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148278 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148280 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148283 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148285 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148288 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148290 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148294 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148298 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148301 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148304 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148307 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148309 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148312 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148315 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148318 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148320 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148323 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148327 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148329 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:08.150346 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148332 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148334 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148337 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148340 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148343 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148346 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148349 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148352 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148355 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148357 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148360 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148363 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148366 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148369 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148371 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148374 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148376 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148379 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148381 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.148384 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:08.151182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150604 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150620 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150626 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150632 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150637 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150642 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150646 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150651 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150655 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150660 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150664 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150669 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150673 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150678 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150682 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150686 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150690 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150694 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150699 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150703 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:08.152042 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150707 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150712 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150716 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150721 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150725 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150729 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150733 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150738 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150742 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150746 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150750 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150755 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150760 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150764 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150768 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150772 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150776 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150781 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150785 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:08.152900 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150791 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150798 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150805 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150811 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150815 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150821 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150828 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150833 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150838 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150843 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150847 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150851 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150855 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150861 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150865 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150870 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150874 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150878 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150883 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:08.153826 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150888 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150892 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150896 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150900 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150905 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150909 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150914 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150919 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150942 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150946 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150951 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150955 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150959 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150963 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150967 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150971 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150975 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150981 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150984 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150989 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:08.154343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150993 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.150998 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151002 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151006 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151010 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151025 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151031 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151035 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151039 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151043 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151047 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151051 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151055 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151059 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151064 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151068 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151072 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151076 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151081 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151085 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:08.154977 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151096 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151744 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151752 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151757 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151761 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151765 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151772 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151777 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151782 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151786 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151790 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151794 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151799 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151803 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151807 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151811 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151815 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151828 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151833 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:08.155570 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151837 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151841 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151846 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151850 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151854 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151859 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151863 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151867 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151871 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151876 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151880 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151884 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151889 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151893 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151897 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151902 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151906 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151911 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151915 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151919 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:08.156182 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151942 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151946 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151951 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151955 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151959 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151963 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151967 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151972 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151975 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151981 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.151986 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152001 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152007 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152011 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152015 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152020 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152024 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152028 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152032 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152036 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:08.156680 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152043 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152048 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152053 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152058 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152062 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152067 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152071 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152076 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152080 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152084 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152091 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152095 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152099 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152104 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152109 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152113 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152117 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152121 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152125 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152130 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:08.157271 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152134 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152138 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152142 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152146 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152159 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152164 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152168 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152173 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152177 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152181 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152188 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152194 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152198 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152203 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152207 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152212 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152222 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152226 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152231 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:08.157779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152236 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.152240 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152338 2578 flags.go:64] FLAG: --address="0.0.0.0" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152348 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152363 2578 flags.go:64] FLAG: --anonymous-auth="true" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152370 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152376 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152383 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152389 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152396 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152401 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152406 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152412 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152417 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152422 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152426 2578 flags.go:64] FLAG: --cgroup-root="" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152431 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152435 2578 flags.go:64] FLAG: --client-ca-file="" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152450 2578 flags.go:64] FLAG: --cloud-provider="external" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152455 2578 flags.go:64] FLAG: --cluster-dns="[]" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152464 2578 flags.go:64] FLAG: --cluster-domain="" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152468 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152473 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152478 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 07 09:08:08.158306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152484 2578 flags.go:64] FLAG: --container-log-max-files="5" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152495 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152500 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152504 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152509 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152516 2578 flags.go:64] FLAG: --contention-profiling="false" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152521 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152526 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152530 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152535 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152542 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152547 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152551 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152556 2578 flags.go:64] FLAG: --enable-load-reader="false" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152561 2578 flags.go:64] FLAG: --enable-server="true" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152568 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152577 2578 flags.go:64] FLAG: --event-burst="100" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152582 2578 flags.go:64] FLAG: --event-qps="50" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152587 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152591 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152596 2578 flags.go:64] FLAG: --eviction-hard="" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152602 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152607 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152612 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152617 2578 flags.go:64] FLAG: --eviction-soft="" Jun 07 09:08:08.158889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152622 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152627 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152633 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152637 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152642 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152646 2578 flags.go:64] FLAG: --fail-swap-on="true" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152651 2578 flags.go:64] FLAG: --feature-gates="" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152657 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152662 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152666 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152671 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152676 2578 flags.go:64] FLAG: --healthz-port="10248" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152684 2578 flags.go:64] FLAG: --help="false" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152688 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152694 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152699 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152704 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152709 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152714 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152719 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152724 2578 flags.go:64] FLAG: --image-service-endpoint="" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152728 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152734 2578 flags.go:64] FLAG: --kube-api-burst="100" Jun 07 09:08:08.159505 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152739 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152745 2578 flags.go:64] FLAG: --kube-api-qps="50" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152749 2578 flags.go:64] FLAG: --kube-reserved="" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152754 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152758 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152763 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152768 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152773 2578 flags.go:64] FLAG: --lock-file="" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152777 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152782 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152787 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152795 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152801 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152805 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152810 2578 flags.go:64] FLAG: --logging-format="text" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152815 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152820 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152825 2578 flags.go:64] FLAG: --manifest-url="" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152829 2578 flags.go:64] FLAG: --manifest-url-header="" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152836 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152841 2578 flags.go:64] FLAG: --max-open-files="1000000" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152848 2578 flags.go:64] FLAG: --max-pods="110" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152853 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152858 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152862 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jun 07 09:08:08.160170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152867 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152872 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152877 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152881 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152893 2578 flags.go:64] FLAG: --node-status-max-images="50" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152898 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152904 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152909 2578 flags.go:64] FLAG: --pod-cidr="" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152913 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152940 2578 flags.go:64] FLAG: --pod-manifest-path="" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152945 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152950 2578 flags.go:64] FLAG: --pods-per-core="0" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152955 2578 flags.go:64] FLAG: --port="10250" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152959 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152964 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08f729a4916ee49d4" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152969 2578 flags.go:64] FLAG: --qos-reserved="" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152974 2578 flags.go:64] FLAG: --read-only-port="10255" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152979 2578 flags.go:64] FLAG: --register-node="true" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152984 2578 flags.go:64] FLAG: --register-with-taints="" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152991 2578 flags.go:64] FLAG: --registry-burst="10" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.152998 2578 flags.go:64] FLAG: --registry-qps="5" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153004 2578 flags.go:64] FLAG: --reserved-cpus="" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153009 2578 flags.go:64] FLAG: --reserved-memory="" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153014 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153019 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 07 09:08:08.160797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153024 2578 flags.go:64] FLAG: --rotate-certificates="false" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153028 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153033 2578 flags.go:64] FLAG: --runonce="false" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153037 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153044 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153049 2578 flags.go:64] FLAG: --seccomp-default="false" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153053 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153058 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153063 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153068 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153073 2578 flags.go:64] FLAG: --storage-driver-password="root" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153078 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153083 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153089 2578 flags.go:64] FLAG: --storage-driver-user="root" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153093 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153098 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153103 2578 flags.go:64] FLAG: --system-cgroups="" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153108 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153117 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153121 2578 flags.go:64] FLAG: --tls-cert-file="" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153126 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153132 2578 flags.go:64] FLAG: --tls-min-version="" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153137 2578 flags.go:64] FLAG: --tls-private-key-file="" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153141 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153146 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 07 09:08:08.161474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153151 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153156 2578 flags.go:64] FLAG: --v="2" Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153162 2578 flags.go:64] FLAG: --version="false" Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153169 2578 flags.go:64] FLAG: --vmodule="" Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153175 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.153180 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153361 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153367 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153372 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153376 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153381 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153387 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153391 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153396 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153400 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153405 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153409 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153413 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153418 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153424 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153432 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:08.162127 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153436 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153441 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153446 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153451 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153456 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153461 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153465 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153469 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153473 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153478 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153482 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153486 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153493 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153499 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153503 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153509 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153513 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153517 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153521 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153527 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:08.162673 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153531 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153536 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153542 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153546 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153550 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153556 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153562 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153567 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153571 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153576 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153580 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153586 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153590 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153595 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153599 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153603 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153607 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153611 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153616 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:08.163280 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153620 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153624 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153628 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153633 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153637 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153641 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153645 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153650 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153654 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153660 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153664 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153669 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153673 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153677 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153681 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153687 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153691 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153695 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153700 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153704 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:08.163760 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153708 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153712 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153716 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153721 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153726 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153740 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153745 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153749 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153754 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153758 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153763 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153767 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153772 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153776 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153780 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153784 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153789 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153793 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153797 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153802 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:08.164302 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153806 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153810 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153815 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153819 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.153823 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.154579 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.161455 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.161472 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.162367 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.162374 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162429 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162433 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162437 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162441 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162444 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:08.164813 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162447 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162450 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162453 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162456 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162458 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162461 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162463 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162466 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162469 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162472 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162475 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162478 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162481 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162484 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162487 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162489 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162492 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162494 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162497 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162500 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:08.165207 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162503 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162505 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162508 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162511 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162515 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162518 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162520 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162523 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162525 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162528 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162531 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162535 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162538 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162541 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162544 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162547 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162550 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162553 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162556 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:08.165695 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162559 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162562 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162565 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162568 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162571 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162574 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162577 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162581 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162585 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162587 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162590 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162592 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162595 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162598 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162601 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162603 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162606 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162609 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162612 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162615 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:08.166181 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162618 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162621 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162623 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162626 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162628 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162631 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162633 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162636 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162638 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162641 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162644 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162646 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162649 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162651 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162654 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162657 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162660 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162662 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162665 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162667 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:08.166660 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162670 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162673 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162675 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162678 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162682 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162686 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162689 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162692 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162695 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162698 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162700 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162703 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162706 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162709 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:08.167149 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162711 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.162716 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162827 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162831 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162834 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162837 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162840 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162843 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162847 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162851 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162854 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162857 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162860 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162863 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:08.167494 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162866 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162868 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162871 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162873 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162876 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162879 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162881 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162884 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162886 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162889 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162891 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162894 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162896 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162899 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162901 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162904 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162911 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162914 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162916 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162919 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:08.167842 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162936 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162941 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162945 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162949 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162951 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162954 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162958 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162961 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162965 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162968 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162971 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162974 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162977 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162979 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162982 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162985 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162988 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162990 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162993 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:08.168411 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162995 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.162998 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163000 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163003 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163005 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163008 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163011 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163013 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163016 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163019 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163022 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163025 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163028 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163030 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163033 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163035 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163038 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163040 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163043 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163046 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:08.168864 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163048 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163051 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163053 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163056 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163058 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163061 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163063 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163066 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163068 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163071 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163073 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163076 2578 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163078 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163081 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163084 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163086 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163089 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163091 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163094 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163096 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:08.169369 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163099 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163102 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163104 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163108 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163110 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163113 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163115 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:08.163118 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.163123 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.163228 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.166160 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.167560 2578 server.go:1019] "Starting client certificate rotation" Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.167662 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 07 09:08:08.169898 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.167733 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 07 09:08:08.170358 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.170345 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 07 09:08:08.174693 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.174669 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 07 09:08:08.186683 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.186663 2578 log.go:25] "Validated CRI v1 runtime API" Jun 07 09:08:08.189813 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.189796 2578 log.go:25] "Validated CRI v1 image API" Jun 07 09:08:08.190528 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.190513 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 07 09:08:08.193185 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.193167 2578 fs.go:135] Filesystem UUIDs: map[1cf6625c-b306-46ce-8bc4-d2d131038687:/dev/nvme0n1p4 4acc5ca5-caa3-4f5d-ad50-ddbf7f94c8c7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jun 07 09:08:08.193246 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.193185 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 07 09:08:08.199018 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.198880 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 07 09:08:08.199180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.199065 2578 manager.go:216] Machine: {Timestamp:2026-06-07 09:08:08.196730295 +0000 UTC m=+0.341791205 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107500 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec242a4c7492205ba8b92620343fee3e SystemUUID:ec242a4c-7492-205b-a8b9-2620343fee3e BootID:7109071b-f61a-4aea-bcb3-6f3628c19cce Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:29:07:be:9f:67 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:29:07:be:9f:67 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:90:57:af:44:8c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 07 09:08:08.199180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.199178 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 07 09:08:08.199333 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.199277 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 07 09:08:08.200424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.200398 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 07 09:08:08.200575 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.200427 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-203.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 07 09:08:08.201574 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.201562 2578 topology_manager.go:138] "Creating topology manager with none policy" Jun 07 09:08:08.201641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.201579 2578 container_manager_linux.go:309] "Creating device plugin manager" Jun 07 09:08:08.201641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.201590 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 07 09:08:08.202573 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.202561 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 07 09:08:08.203387 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.203375 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 07 09:08:08.206489 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.206476 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 07 09:08:08.206621 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.206610 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 07 09:08:08.206689 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.206645 2578 kubelet.go:486] "Attempting to sync node with API server" Jun 07 09:08:08.206689 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.206659 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 07 09:08:08.206689 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.206678 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 07 09:08:08.206809 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.206692 2578 kubelet.go:398] "Adding apiserver pod source" Jun 07 09:08:08.206809 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.206704 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 07 09:08:08.207097 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.207083 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 07 09:08:08.207167 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.207108 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 07 09:08:08.209848 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.209827 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 07 09:08:08.211143 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.211127 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 07 09:08:08.211212 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.211160 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 07 09:08:08.211812 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.211800 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 07 09:08:08.211864 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.211816 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 07 09:08:08.211864 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.211822 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 07 09:08:08.211864 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.211828 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 07 09:08:08.211864 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.211836 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 07 09:08:08.211864 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.211841 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 07 09:08:08.212478 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.212469 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 07 09:08:08.212478 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.212477 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 07 09:08:08.212533 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.212485 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 07 09:08:08.212533 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.212490 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 07 09:08:08.212533 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.212498 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 07 09:08:08.212533 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.212507 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 07 09:08:08.213953 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.213942 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 07 09:08:08.213999 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.213954 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 07 09:08:08.216836 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.216782 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-203.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 07 09:08:08.216943 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.216838 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 07 09:08:08.216943 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.216849 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 07 09:08:08.217197 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.217181 2578 server.go:1268] "Started kubelet" Jun 07 09:08:08.218045 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.218003 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 07 09:08:08.218143 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.218082 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 07 09:08:08.218185 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.218167 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 07 09:08:08.218572 ip-10-0-129-203 systemd[1]: Started Kubernetes Kubelet. Jun 07 09:08:08.220248 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.220230 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 07 09:08:08.220597 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.220580 2578 server.go:310] "Adding debug handlers to kubelet server" Jun 07 09:08:08.225259 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.225241 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 07 09:08:08.225335 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.225263 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 07 09:08:08.225394 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.225362 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:08.225909 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.225849 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 07 09:08:08.225909 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.225867 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 07 09:08:08.225909 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.225894 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 07 09:08:08.226094 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.225912 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 07 09:08:08.226094 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226010 2578 reconstruct.go:97] "Volume reconstruction finished" Jun 07 09:08:08.226094 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226018 2578 reconciler.go:29] "Reconciler: start to sync state" Jun 07 09:08:08.226479 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226457 2578 factory.go:55] Registering systemd factory Jun 07 09:08:08.226479 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226480 2578 factory.go:223] Registration of the systemd container factory successfully Jun 07 09:08:08.226693 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226678 2578 factory.go:153] Registering CRI-O factory Jun 07 09:08:08.226755 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226697 2578 factory.go:223] Registration of the crio container factory successfully Jun 07 09:08:08.226755 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226749 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 07 09:08:08.226826 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226774 2578 factory.go:103] Registering Raw factory Jun 07 09:08:08.226826 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.226788 2578 manager.go:1196] Started watching for new ooms in manager Jun 07 09:08:08.227225 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.227213 2578 manager.go:349] Starting recovery of all containers Jun 07 09:08:08.227541 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.226375 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-203.ec2.internal.18b6c1586decec16 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-203.ec2.internal,UID:ip-10-0-129-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-203.ec2.internal,},FirstTimestamp:2026-06-07 09:08:08.21713615 +0000 UTC m=+0.362197065,LastTimestamp:2026-06-07 09:08:08.21713615 +0000 UTC m=+0.362197065,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-203.ec2.internal,}" Jun 07 09:08:08.234869 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.234841 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 07 09:08:08.234989 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.234874 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 07 09:08:08.237150 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.237134 2578 manager.go:354] Recovery completed Jun 07 09:08:08.241304 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.241293 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:08.242962 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.242940 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:08.243025 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.242974 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:08.243025 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.242984 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:08.243460 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.243446 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 07 09:08:08.243460 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.243458 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 07 09:08:08.243583 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.243462 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:08.243583 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.243472 2578 state_mem.go:36] "Initialized new in-memory state store" Jun 07 09:08:08.244121 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.244108 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:08.244215 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.244134 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:08.244215 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.244146 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:08.244734 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.244719 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lk896" Jun 07 09:08:08.245239 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.245226 2578 policy_none.go:49] "None policy: Start" Jun 07 09:08:08.245294 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.245250 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 07 09:08:08.245294 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.245264 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 07 09:08:08.246569 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.246557 2578 policy_none.go:47] "Start" Jun 07 09:08:08.250861 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.250837 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lk896" Jun 07 09:08:08.252976 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.252881 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-203.ec2.internal.18b6c1586f76fcbd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-203.ec2.internal,UID:ip-10-0-129-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-203.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-203.ec2.internal,},FirstTimestamp:2026-06-07 09:08:08.242961597 +0000 UTC m=+0.388022506,LastTimestamp:2026-06-07 09:08:08.242961597 +0000 UTC m=+0.388022506,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-203.ec2.internal,}" Jun 07 09:08:08.281595 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.281580 2578 manager.go:337] "Starting Device Plugin manager" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.281613 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.281626 2578 server.go:87] "Starting device plugin registration server" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.281833 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.281857 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.281958 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.282041 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.282050 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.282512 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.282556 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.291455 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.292603 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.292633 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.292649 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.292681 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 07 09:08:08.297610 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.294097 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:08.382990 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.382962 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:08.385616 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.385598 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:08.385722 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.385627 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:08.385722 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.385637 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:08.385722 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.385662 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.393323 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.393265 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal"] Jun 07 09:08:08.393394 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.393327 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:08.394155 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.394141 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:08.394241 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.394176 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:08.394241 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.394190 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:08.394445 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.394428 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.394505 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.394448 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-203.ec2.internal\": node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:08.395549 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.395534 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:08.395651 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.395639 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.395702 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.395665 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:08.398334 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.398318 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:08.398424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.398343 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:08.398424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.398352 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:08.399151 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.399130 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:08.399237 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.399158 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:08.399237 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.399172 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:08.400728 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.400713 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.400812 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.400741 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:08.401376 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.401361 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:08.401458 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.401386 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:08.401458 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.401400 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:08.416281 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.416265 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:08.416477 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.416464 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-203.ec2.internal\" not found" node="ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.420398 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.420381 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-203.ec2.internal\" not found" node="ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.516622 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.516592 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:08.527979 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.527953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7476c89bb59e907acdd0c6bfd86a930b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"7476c89bb59e907acdd0c6bfd86a930b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.528073 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.527987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7476c89bb59e907acdd0c6bfd86a930b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"7476c89bb59e907acdd0c6bfd86a930b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.528073 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.528011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62ac29cdc11a60617fbf64ef64f4afc5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-203.ec2.internal\" (UID: \"62ac29cdc11a60617fbf64ef64f4afc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.617135 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.617106 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:08.628428 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.628403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7476c89bb59e907acdd0c6bfd86a930b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"7476c89bb59e907acdd0c6bfd86a930b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.628514 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.628436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7476c89bb59e907acdd0c6bfd86a930b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"7476c89bb59e907acdd0c6bfd86a930b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.628514 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.628458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62ac29cdc11a60617fbf64ef64f4afc5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-203.ec2.internal\" (UID: \"62ac29cdc11a60617fbf64ef64f4afc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.628514 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.628506 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7476c89bb59e907acdd0c6bfd86a930b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"7476c89bb59e907acdd0c6bfd86a930b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.628618 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.628507 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62ac29cdc11a60617fbf64ef64f4afc5-config\") pod \"kube-apiserver-proxy-ip-10-0-129-203.ec2.internal\" (UID: \"62ac29cdc11a60617fbf64ef64f4afc5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.628618 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.628510 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7476c89bb59e907acdd0c6bfd86a930b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal\" (UID: \"7476c89bb59e907acdd0c6bfd86a930b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.717942 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.717838 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:08.720996 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.720976 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.725070 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:08.725054 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" Jun 07 09:08:08.818202 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.818172 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:08.918687 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:08.918659 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.019228 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:09.019196 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.119768 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:09.119735 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.167269 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.167242 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 07 09:08:09.167902 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.167418 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 07 09:08:09.220344 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:09.220316 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.246593 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:09.246566 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ac29cdc11a60617fbf64ef64f4afc5.slice/crio-01295980839aa56efbf6351ea22075c920e9e9fe69d7dfffeeb668adce28dd63 WatchSource:0}: Error finding container 01295980839aa56efbf6351ea22075c920e9e9fe69d7dfffeeb668adce28dd63: Status 404 returned error can't find the container with id 01295980839aa56efbf6351ea22075c920e9e9fe69d7dfffeeb668adce28dd63 Jun 07 09:08:09.247065 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:09.247043 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7476c89bb59e907acdd0c6bfd86a930b.slice/crio-520d65a9fe5d4eec6187f268ea510fc608c0f8fa12a2179e935d0c2e0caddf1f WatchSource:0}: Error finding container 520d65a9fe5d4eec6187f268ea510fc608c0f8fa12a2179e935d0c2e0caddf1f: Status 404 returned error can't find the container with id 520d65a9fe5d4eec6187f268ea510fc608c0f8fa12a2179e935d0c2e0caddf1f Jun 07 09:08:09.251299 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.251286 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 07 09:08:09.253997 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.253971 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-06 09:03:08 +0000 UTC" deadline="2027-11-30 19:01:43.200508129 +0000 UTC" Jun 07 09:08:09.253997 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.253994 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12993h53m33.946517633s" Jun 07 09:08:09.295980 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.295901 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" event={"ID":"62ac29cdc11a60617fbf64ef64f4afc5","Type":"ContainerStarted","Data":"01295980839aa56efbf6351ea22075c920e9e9fe69d7dfffeeb668adce28dd63"} Jun 07 09:08:09.296861 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.296841 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" event={"ID":"7476c89bb59e907acdd0c6bfd86a930b","Type":"ContainerStarted","Data":"520d65a9fe5d4eec6187f268ea510fc608c0f8fa12a2179e935d0c2e0caddf1f"} Jun 07 09:08:09.321004 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:09.320979 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.330623 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.330607 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:09.338910 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.338891 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:09.403177 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.403149 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:09.404142 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.404127 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:09.404208 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.404157 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:09.404208 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.404167 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:09.421366 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:09.421344 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.521881 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:09.521839 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.622380 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:09.622311 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.723078 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:09.723041 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-203.ec2.internal\" not found" Jun 07 09:08:09.747452 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.747424 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:09.825833 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.825805 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" Jun 07 09:08:09.837963 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.837917 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 07 09:08:09.839115 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.839094 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" Jun 07 09:08:09.845445 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:09.845422 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 07 09:08:10.208514 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.208487 2578 apiserver.go:52] "Watching apiserver" Jun 07 09:08:10.217308 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.217284 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 07 09:08:10.219885 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.219856 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-msxgz","openshift-multus/network-metrics-daemon-p4shp","openshift-network-diagnostics/network-check-target-b7fds","openshift-ovn-kubernetes/ovnkube-node-6tljf","kube-system/konnectivity-agent-xbns8","kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d","openshift-dns/node-resolver-dfktq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal","openshift-multus/multus-additional-cni-plugins-dvckp","openshift-multus/multus-cv9fw","openshift-network-operator/iptables-alerter-vqv6d","kube-system/global-pull-secret-syncer-n4kfk","openshift-cluster-node-tuning-operator/tuned-n7kht"] Jun 07 09:08:10.221746 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.221618 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.223077 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.222916 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.223077 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.223014 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:10.224084 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.224065 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:10.224185 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.224130 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:10.224952 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.224752 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 07 09:08:10.225058 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.225015 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 07 09:08:10.225436 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.225252 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 07 09:08:10.225436 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.225285 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jsmkt\"" Jun 07 09:08:10.225636 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.225621 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.226769 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.226672 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 07 09:08:10.227182 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.227162 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:10.229213 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.228319 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.229213 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.228884 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7khqb\"" Jun 07 09:08:10.229213 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.229079 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 07 09:08:10.229213 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.228885 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 07 09:08:10.229727 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.229577 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 07 09:08:10.229786 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.229730 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 07 09:08:10.229837 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.229807 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 07 09:08:10.229907 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.229890 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 07 09:08:10.230310 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.230109 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tml4x\"" Jun 07 09:08:10.230310 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.230152 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 07 09:08:10.230310 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.230169 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 07 09:08:10.230871 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.230536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.231219 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.231201 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jnx7z\"" Jun 07 09:08:10.231422 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.231408 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:08:10.233059 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.232154 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 07 09:08:10.234948 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.233524 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 07 09:08:10.234948 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.233524 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mcfzd\"" Jun 07 09:08:10.234948 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.233622 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 07 09:08:10.234948 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.234105 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.236980 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237075 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237194 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5nrcm\"" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237263 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237441 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovn-node-metrics-cert\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237456 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8trx\" (UniqueName: \"kubernetes.io/projected/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-kube-api-access-p8trx\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237478 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-socket-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-registration-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-run-netns\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.237519 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-etc-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237549 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-log-socket\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237592 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7102b2e3-1052-46e4-8f26-e9da44934255-agent-certs\") pod \"konnectivity-agent-xbns8\" (UID: \"7102b2e3-1052-46e4-8f26-e9da44934255\") " pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237613 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-ovn\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-node-log\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7102b2e3-1052-46e4-8f26-e9da44934255-konnectivity-ca\") pod \"konnectivity-agent-xbns8\" (UID: \"7102b2e3-1052-46e4-8f26-e9da44934255\") " pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:10.237845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-cni-netd\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237783 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-env-overrides\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237823 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-device-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237850 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2slc\" (UniqueName: \"kubernetes.io/projected/33c3dd79-440c-41ca-84da-7138d137be32-kube-api-access-x2slc\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38f9e64-890b-4696-99fd-2ca38d35a05d-kubelet-config\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237901 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-var-lib-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-etc-selinux\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.237989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238036 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-sys-fs\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38f9e64-890b-4696-99fd-2ca38d35a05d-dbus\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovnkube-script-lib\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238091 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-kubelet\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.238774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238104 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-systemd\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.239674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-cni-bin\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.239674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238129 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-systemd-units\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.239674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-slash\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.239674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238184 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovnkube-config\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.239674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238541 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.239674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.238958 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 07 09:08:10.240427 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.240156 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.240759 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.240672 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 07 09:08:10.241140 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.241122 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m9x2v\"" Jun 07 09:08:10.241455 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.241440 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 07 09:08:10.241567 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.241548 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 07 09:08:10.241632 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.241553 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.242082 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.241759 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 07 09:08:10.242082 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.241863 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 07 09:08:10.242576 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.242555 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 07 09:08:10.242777 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.242758 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wmwzk\"" Jun 07 09:08:10.243782 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.243766 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pjz5b\"" Jun 07 09:08:10.244055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.244038 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:08:10.244330 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.244311 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 07 09:08:10.244420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.244340 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 07 09:08:10.261546 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.261523 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rhsn2" Jun 07 09:08:10.269145 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.269126 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rhsn2" Jun 07 09:08:10.327939 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.327892 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 07 09:08:10.338914 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.338889 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djznd\" (UniqueName: \"kubernetes.io/projected/795d8080-8304-4fe8-822b-723fc3feafbe-kube-api-access-djznd\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.339067 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.338936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-iptables-alerter-script\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.339067 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.338961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9v98\" (UniqueName: \"kubernetes.io/projected/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-kube-api-access-x9v98\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.339067 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-device-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.339211 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339110 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-device-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.339211 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-var-lib-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.339211 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-system-cni-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.339211 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-cni-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.339211 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-socket-dir-parent\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.339424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/795d8080-8304-4fe8-822b-723fc3feafbe-serviceca\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.339424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339253 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-etc-selinux\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.339424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-run\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.339424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-var-lib-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.339424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-host-slash\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.339424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339338 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-etc-selinux\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.339424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339390 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd4fv\" (UniqueName: \"kubernetes.io/projected/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-kube-api-access-sd4fv\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.339424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339418 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vjd4\" (UniqueName: \"kubernetes.io/projected/dfbdde5e-d359-422e-bf75-55f3221bd8f3-kube-api-access-5vjd4\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-sys-fs\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339514 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfnlh\" (UniqueName: \"kubernetes.io/projected/dbb55607-63d7-4a8c-b528-e37853d67387-kube-api-access-cfnlh\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-hosts-file\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-systemd\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-cnibin\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339595 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-sys-fs\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-systemd-units\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339616 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-systemd\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-slash\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339652 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-slash\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-cnibin\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339673 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-systemd-units\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339686 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-cni-bin\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795d8080-8304-4fe8-822b-723fc3feafbe-host\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339729 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysconfig\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.339764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7102b2e3-1052-46e4-8f26-e9da44934255-agent-certs\") pod \"konnectivity-agent-xbns8\" (UID: \"7102b2e3-1052-46e4-8f26-e9da44934255\") " pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-os-release\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339873 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-node-log\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-k8s-cni-cncf-io\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.339984 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-daemon-config\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340005 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-var-lib-kubelet\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340027 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dbb55607-63d7-4a8c-b528-e37853d67387-etc-tuned\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340083 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-node-log\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-cni-netd\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340173 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340204 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-conf-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340206 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-cni-netd\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.340619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg88p\" (UniqueName: \"kubernetes.io/projected/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-kube-api-access-sg88p\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340223 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2slc\" (UniqueName: \"kubernetes.io/projected/33c3dd79-440c-41ca-84da-7138d137be32-kube-api-access-x2slc\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340367 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38f9e64-890b-4696-99fd-2ca38d35a05d-kubelet-config\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340384 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-kubernetes\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340397 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysctl-d\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-sys\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-system-cni-dir\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340470 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38f9e64-890b-4696-99fd-2ca38d35a05d-kubelet-config\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-os-release\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340528 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-systemd\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340588 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-run-ovn-kubernetes\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.341402 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.340639 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.340720 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret podName:e38f9e64-890b-4696-99fd-2ca38d35a05d nodeName:}" failed. No retries permitted until 2026-06-07 09:08:10.840686399 +0000 UTC m=+2.985747317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret") pod "global-pull-secret-syncer-n4kfk" (UID: "e38f9e64-890b-4696-99fd-2ca38d35a05d") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340740 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38f9e64-890b-4696-99fd-2ca38d35a05d-dbus\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovnkube-script-lib\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340794 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-kubelet\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-cni-bin\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbb55607-63d7-4a8c-b528-e37853d67387-tmp\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340865 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-tmp-dir\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340880 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-kubelet\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340889 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340906 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38f9e64-890b-4696-99fd-2ca38d35a05d-dbus\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnfqd\" (UniqueName: \"kubernetes.io/projected/8904cd6d-8eaf-4852-8672-9e1651cab35f-kube-api-access-mnfqd\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.340983 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-etc-kubernetes\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovnkube-config\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341041 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-cni-bin\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovn-node-metrics-cert\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8trx\" (UniqueName: \"kubernetes.io/projected/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-kube-api-access-p8trx\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-kubelet\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-socket-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-registration-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-run-netns\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-etc-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341408 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-log-socket\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341415 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-socket-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341435 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysctl-conf\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-multus-certs\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-run-netns\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-modprobe-d\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341486 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/33c3dd79-440c-41ca-84da-7138d137be32-registration-dir\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341509 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-log-socket\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-ovn\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341526 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovnkube-script-lib\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341556 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7102b2e3-1052-46e4-8f26-e9da44934255-konnectivity-ca\") pod \"konnectivity-agent-xbns8\" (UID: \"7102b2e3-1052-46e4-8f26-e9da44934255\") " pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:10.342818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341607 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-run-ovn\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-cni-binary-copy\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-cni-multus\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341700 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-etc-openvswitch\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-hostroot\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341749 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-host\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341809 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-env-overrides\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341835 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-netns\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341841 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.341860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-lib-modules\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.342044 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovnkube-config\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.342122 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7102b2e3-1052-46e4-8f26-e9da44934255-konnectivity-ca\") pod \"konnectivity-agent-xbns8\" (UID: \"7102b2e3-1052-46e4-8f26-e9da44934255\") " pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:10.343378 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.342242 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-env-overrides\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.343843 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.343384 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7102b2e3-1052-46e4-8f26-e9da44934255-agent-certs\") pod \"konnectivity-agent-xbns8\" (UID: \"7102b2e3-1052-46e4-8f26-e9da44934255\") " pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:10.344559 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.344540 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-ovn-node-metrics-cert\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.345102 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.345085 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:10.345190 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.345106 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:10.345190 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.345119 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zjs77 for pod openshift-network-diagnostics/network-check-target-b7fds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:10.345190 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.345175 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77 podName:77afbb38-7007-42a8-8d2e-6203b109a0a0 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:10.84515945 +0000 UTC m=+2.990220366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zjs77" (UniqueName: "kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77") pod "network-check-target-b7fds" (UID: "77afbb38-7007-42a8-8d2e-6203b109a0a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:10.348395 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.348377 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2slc\" (UniqueName: \"kubernetes.io/projected/33c3dd79-440c-41ca-84da-7138d137be32-kube-api-access-x2slc\") pod \"aws-ebs-csi-driver-node-lnl9d\" (UID: \"33c3dd79-440c-41ca-84da-7138d137be32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.349024 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.349005 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8trx\" (UniqueName: \"kubernetes.io/projected/d5cd375f-3dde-4e82-a91e-fe1b006c0cc9-kube-api-access-p8trx\") pod \"ovnkube-node-6tljf\" (UID: \"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9\") " pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.442451 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnfqd\" (UniqueName: \"kubernetes.io/projected/8904cd6d-8eaf-4852-8672-9e1651cab35f-kube-api-access-mnfqd\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-etc-kubernetes\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-kubelet\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysctl-conf\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-multus-certs\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-modprobe-d\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-cni-binary-copy\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-cni-multus\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.442622 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-hostroot\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-host\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-netns\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442662 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-etc-kubernetes\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442684 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-kubelet\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442676 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442729 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-hostroot\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442734 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-host\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442744 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysctl-conf\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-lib-modules\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442730 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-cni-multus\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442763 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-netns\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442761 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-multus-certs\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djznd\" (UniqueName: \"kubernetes.io/projected/795d8080-8304-4fe8-822b-723fc3feafbe-kube-api-access-djznd\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-iptables-alerter-script\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442863 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-modprobe-d\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442882 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-lib-modules\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9v98\" (UniqueName: \"kubernetes.io/projected/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-kube-api-access-x9v98\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.443068 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-system-cni-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.442992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-cni-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-socket-dir-parent\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/795d8080-8304-4fe8-822b-723fc3feafbe-serviceca\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-run\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443045 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-system-cni-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443054 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-host-slash\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443073 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd4fv\" (UniqueName: \"kubernetes.io/projected/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-kube-api-access-sd4fv\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443083 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-socket-dir-parent\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443088 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vjd4\" (UniqueName: \"kubernetes.io/projected/dfbdde5e-d359-422e-bf75-55f3221bd8f3-kube-api-access-5vjd4\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443139 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-host-slash\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443140 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-cni-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfnlh\" (UniqueName: \"kubernetes.io/projected/dbb55607-63d7-4a8c-b528-e37853d67387-kube-api-access-cfnlh\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443192 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-hosts-file\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443219 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-cnibin\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443241 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-cni-binary-copy\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443245 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-cnibin\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443284 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-cnibin\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.443863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-cni-bin\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795d8080-8304-4fe8-822b-723fc3feafbe-host\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443326 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-hosts-file\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-run\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysconfig\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443365 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-cnibin\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-os-release\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443387 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-var-lib-cni-bin\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443403 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795d8080-8304-4fe8-822b-723fc3feafbe-host\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-k8s-cni-cncf-io\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443450 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysconfig\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-os-release\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-daemon-config\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443473 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/795d8080-8304-4fe8-822b-723fc3feafbe-serviceca\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443484 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-var-lib-kubelet\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443483 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-host-run-k8s-cni-cncf-io\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dbb55607-63d7-4a8c-b528-e37853d67387-etc-tuned\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.444678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443516 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-iptables-alerter-script\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-conf-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg88p\" (UniqueName: \"kubernetes.io/projected/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-kube-api-access-sg88p\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443603 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-var-lib-kubelet\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.443668 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443678 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-conf-dir\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-kubernetes\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.443866 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:10.943846706 +0000 UTC m=+3.088907608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.443981 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-kubernetes\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444018 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-cni-binary-copy\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444078 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-multus-daemon-config\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444110 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysctl-d\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-sys\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-system-cni-dir\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.445532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444180 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-os-release\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-systemd\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444260 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-os-release\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-sysctl-d\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-sys\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbb55607-63d7-4a8c-b528-e37853d67387-tmp\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444310 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dbb55607-63d7-4a8c-b528-e37853d67387-etc-systemd\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444317 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8904cd6d-8eaf-4852-8672-9e1651cab35f-system-cni-dir\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-tmp-dir\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444421 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8904cd6d-8eaf-4852-8672-9e1651cab35f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.444663 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-tmp-dir\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.446303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.446273 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dbb55607-63d7-4a8c-b528-e37853d67387-etc-tuned\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.446756 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.446532 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbb55607-63d7-4a8c-b528-e37853d67387-tmp\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.451130 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.451100 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnfqd\" (UniqueName: \"kubernetes.io/projected/8904cd6d-8eaf-4852-8672-9e1651cab35f-kube-api-access-mnfqd\") pod \"multus-additional-cni-plugins-dvckp\" (UID: \"8904cd6d-8eaf-4852-8672-9e1651cab35f\") " pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.453432 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.453384 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfnlh\" (UniqueName: \"kubernetes.io/projected/dbb55607-63d7-4a8c-b528-e37853d67387-kube-api-access-cfnlh\") pod \"tuned-n7kht\" (UID: \"dbb55607-63d7-4a8c-b528-e37853d67387\") " pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.453726 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.453705 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vjd4\" (UniqueName: \"kubernetes.io/projected/dfbdde5e-d359-422e-bf75-55f3221bd8f3-kube-api-access-5vjd4\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:10.453936 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.453902 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djznd\" (UniqueName: \"kubernetes.io/projected/795d8080-8304-4fe8-822b-723fc3feafbe-kube-api-access-djznd\") pod \"node-ca-msxgz\" (UID: \"795d8080-8304-4fe8-822b-723fc3feafbe\") " pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.454347 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.454328 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9v98\" (UniqueName: \"kubernetes.io/projected/ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4-kube-api-access-x9v98\") pod \"iptables-alerter-vqv6d\" (UID: \"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4\") " pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.454479 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.454461 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd4fv\" (UniqueName: \"kubernetes.io/projected/d7f74ed1-5a63-4261-b482-e4a9e2b2e47b-kube-api-access-sd4fv\") pod \"node-resolver-dfktq\" (UID: \"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b\") " pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.454611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.454592 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg88p\" (UniqueName: \"kubernetes.io/projected/e47627b3-c7bf-4e90-9ee8-69919c6b8c13-kube-api-access-sg88p\") pod \"multus-cv9fw\" (UID: \"e47627b3-c7bf-4e90-9ee8-69919c6b8c13\") " pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.463595 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.463552 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:10.533815 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.533781 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" Jun 07 09:08:10.543649 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.543623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:10.552474 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.552450 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:10.558363 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.558344 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n7kht" Jun 07 09:08:10.566062 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.566043 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dfktq" Jun 07 09:08:10.573641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.573616 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-msxgz" Jun 07 09:08:10.579861 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.579842 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dvckp" Jun 07 09:08:10.587656 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.587637 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cv9fw" Jun 07 09:08:10.592350 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.592331 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vqv6d" Jun 07 09:08:10.847302 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.847273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:10.847460 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.847316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:10.847515 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.847456 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:10.847554 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.847516 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:10.847554 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.847537 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret podName:e38f9e64-890b-4696-99fd-2ca38d35a05d nodeName:}" failed. No retries permitted until 2026-06-07 09:08:11.847514636 +0000 UTC m=+3.992575553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret") pod "global-pull-secret-syncer-n4kfk" (UID: "e38f9e64-890b-4696-99fd-2ca38d35a05d") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:10.847554 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.847540 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:10.847658 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.847563 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zjs77 for pod openshift-network-diagnostics/network-check-target-b7fds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:10.847658 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.847602 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77 podName:77afbb38-7007-42a8-8d2e-6203b109a0a0 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:11.847591358 +0000 UTC m=+3.992652257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjs77" (UniqueName: "kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77") pod "network-check-target-b7fds" (UID: "77afbb38-7007-42a8-8d2e-6203b109a0a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:10.948574 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:10.948541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:10.948744 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.948672 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:10.948744 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:10.948736 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:11.948717083 +0000 UTC m=+4.093777986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:11.126137 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.126051 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cd375f_3dde_4e82_a91e_fe1b006c0cc9.slice/crio-a06aa3420f08a0972a35317352bfafd9800784535343475a8a88faefff08d8b1 WatchSource:0}: Error finding container a06aa3420f08a0972a35317352bfafd9800784535343475a8a88faefff08d8b1: Status 404 returned error can't find the container with id a06aa3420f08a0972a35317352bfafd9800784535343475a8a88faefff08d8b1 Jun 07 09:08:11.128087 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.128058 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7102b2e3_1052_46e4_8f26_e9da44934255.slice/crio-79897e0f77945f56c3648a8cf5574ea1e5832e19e2ab3b40b99aedcbf9287beb WatchSource:0}: Error finding container 79897e0f77945f56c3648a8cf5574ea1e5832e19e2ab3b40b99aedcbf9287beb: Status 404 returned error can't find the container with id 79897e0f77945f56c3648a8cf5574ea1e5832e19e2ab3b40b99aedcbf9287beb Jun 07 09:08:11.128823 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.128794 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795d8080_8304_4fe8_822b_723fc3feafbe.slice/crio-71415386a6c39ff630bd2c4cedce71bcb85dacef4f27478d11a10b311e8ff2b6 WatchSource:0}: Error finding container 71415386a6c39ff630bd2c4cedce71bcb85dacef4f27478d11a10b311e8ff2b6: Status 404 returned error can't find the container with id 71415386a6c39ff630bd2c4cedce71bcb85dacef4f27478d11a10b311e8ff2b6 Jun 07 09:08:11.129990 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.129965 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb55607_63d7_4a8c_b528_e37853d67387.slice/crio-154f7869a5b839bacb62afd4d9c4b21216f43a53fe8e39076cfecc46bc2527b3 WatchSource:0}: Error finding container 154f7869a5b839bacb62afd4d9c4b21216f43a53fe8e39076cfecc46bc2527b3: Status 404 returned error can't find the container with id 154f7869a5b839bacb62afd4d9c4b21216f43a53fe8e39076cfecc46bc2527b3 Jun 07 09:08:11.131675 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.131504 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c3dd79_440c_41ca_84da_7138d137be32.slice/crio-98990dae3470519ee8c67f65af943de075b5f6f984e213275ebc992bb6b25071 WatchSource:0}: Error finding container 98990dae3470519ee8c67f65af943de075b5f6f984e213275ebc992bb6b25071: Status 404 returned error can't find the container with id 98990dae3470519ee8c67f65af943de075b5f6f984e213275ebc992bb6b25071 Jun 07 09:08:11.134743 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.134725 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f74ed1_5a63_4261_b482_e4a9e2b2e47b.slice/crio-3d0695e5ba3c4772f07809844180e5417bed4aeddfbaaf436fe9399a84d705c3 WatchSource:0}: Error finding container 3d0695e5ba3c4772f07809844180e5417bed4aeddfbaaf436fe9399a84d705c3: Status 404 returned error can't find the container with id 3d0695e5ba3c4772f07809844180e5417bed4aeddfbaaf436fe9399a84d705c3 Jun 07 09:08:11.135843 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.135590 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8904cd6d_8eaf_4852_8672_9e1651cab35f.slice/crio-02e068a03f44828dace687d69ce5c3dd3f8bfd535094b496b0f91c3e419b6c10 WatchSource:0}: Error finding container 02e068a03f44828dace687d69ce5c3dd3f8bfd535094b496b0f91c3e419b6c10: Status 404 returned error can't find the container with id 02e068a03f44828dace687d69ce5c3dd3f8bfd535094b496b0f91c3e419b6c10 Jun 07 09:08:11.136362 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.136342 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode47627b3_c7bf_4e90_9ee8_69919c6b8c13.slice/crio-7d72999856379bd22f865d6da2a34c3a512f07b4672d54d68294806673d2e9e0 WatchSource:0}: Error finding container 7d72999856379bd22f865d6da2a34c3a512f07b4672d54d68294806673d2e9e0: Status 404 returned error can't find the container with id 7d72999856379bd22f865d6da2a34c3a512f07b4672d54d68294806673d2e9e0 Jun 07 09:08:11.137274 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:11.137196 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba9326ba_4de3_4c15_91dd_d7eb6fa9b1c4.slice/crio-ffb1dc84c3ed1b9e1658f4eb1be7a1bfc7bb687c99a9af52768e88d331e5ae45 WatchSource:0}: Error finding container ffb1dc84c3ed1b9e1658f4eb1be7a1bfc7bb687c99a9af52768e88d331e5ae45: Status 404 returned error can't find the container with id ffb1dc84c3ed1b9e1658f4eb1be7a1bfc7bb687c99a9af52768e88d331e5ae45 Jun 07 09:08:11.271017 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.270903 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-06 09:03:10 +0000 UTC" deadline="2027-12-16 22:11:04.486099697 +0000 UTC" Jun 07 09:08:11.271017 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.271017 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13381h2m53.215088183s" Jun 07 09:08:11.300984 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.300954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cv9fw" event={"ID":"e47627b3-c7bf-4e90-9ee8-69919c6b8c13","Type":"ContainerStarted","Data":"7d72999856379bd22f865d6da2a34c3a512f07b4672d54d68294806673d2e9e0"} Jun 07 09:08:11.302045 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.302023 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dfktq" event={"ID":"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b","Type":"ContainerStarted","Data":"3d0695e5ba3c4772f07809844180e5417bed4aeddfbaaf436fe9399a84d705c3"} Jun 07 09:08:11.302957 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.302936 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xbns8" event={"ID":"7102b2e3-1052-46e4-8f26-e9da44934255","Type":"ContainerStarted","Data":"79897e0f77945f56c3648a8cf5574ea1e5832e19e2ab3b40b99aedcbf9287beb"} Jun 07 09:08:11.304678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.304654 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" event={"ID":"62ac29cdc11a60617fbf64ef64f4afc5","Type":"ContainerStarted","Data":"677c0018cb1f909b63ca161401e75291983dde4cfdb71f1755b8c183aac21cc4"} Jun 07 09:08:11.305858 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.305834 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vqv6d" event={"ID":"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4","Type":"ContainerStarted","Data":"ffb1dc84c3ed1b9e1658f4eb1be7a1bfc7bb687c99a9af52768e88d331e5ae45"} Jun 07 09:08:11.306857 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.306839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dvckp" event={"ID":"8904cd6d-8eaf-4852-8672-9e1651cab35f","Type":"ContainerStarted","Data":"02e068a03f44828dace687d69ce5c3dd3f8bfd535094b496b0f91c3e419b6c10"} Jun 07 09:08:11.307898 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.307879 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" event={"ID":"33c3dd79-440c-41ca-84da-7138d137be32","Type":"ContainerStarted","Data":"98990dae3470519ee8c67f65af943de075b5f6f984e213275ebc992bb6b25071"} Jun 07 09:08:11.308967 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.308945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n7kht" event={"ID":"dbb55607-63d7-4a8c-b528-e37853d67387","Type":"ContainerStarted","Data":"154f7869a5b839bacb62afd4d9c4b21216f43a53fe8e39076cfecc46bc2527b3"} Jun 07 09:08:11.309868 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.309850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-msxgz" event={"ID":"795d8080-8304-4fe8-822b-723fc3feafbe","Type":"ContainerStarted","Data":"71415386a6c39ff630bd2c4cedce71bcb85dacef4f27478d11a10b311e8ff2b6"} Jun 07 09:08:11.312210 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.312189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"a06aa3420f08a0972a35317352bfafd9800784535343475a8a88faefff08d8b1"} Jun 07 09:08:11.319277 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.319231 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-203.ec2.internal" podStartSLOduration=2.319219693 podStartE2EDuration="2.319219693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:11.318950004 +0000 UTC m=+3.464010922" watchObservedRunningTime="2026-06-07 09:08:11.319219693 +0000 UTC m=+3.464280679" Jun 07 09:08:11.855287 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.855198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:11.855479 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.855336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:11.855555 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:11.855490 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:11.855555 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:11.855511 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:11.855555 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:11.855523 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zjs77 for pod openshift-network-diagnostics/network-check-target-b7fds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:11.855696 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:11.855585 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77 podName:77afbb38-7007-42a8-8d2e-6203b109a0a0 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:13.855568205 +0000 UTC m=+6.000629107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjs77" (UniqueName: "kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77") pod "network-check-target-b7fds" (UID: "77afbb38-7007-42a8-8d2e-6203b109a0a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:11.856037 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:11.856017 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:11.856118 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:11.856074 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret podName:e38f9e64-890b-4696-99fd-2ca38d35a05d nodeName:}" failed. No retries permitted until 2026-06-07 09:08:13.856059721 +0000 UTC m=+6.001120623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret") pod "global-pull-secret-syncer-n4kfk" (UID: "e38f9e64-890b-4696-99fd-2ca38d35a05d") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:11.956466 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:11.956429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:11.956653 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:11.956569 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:11.956653 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:11.956634 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:13.956616639 +0000 UTC m=+6.101677541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:12.271304 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:12.271257 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-06 09:03:10 +0000 UTC" deadline="2028-01-03 17:46:15.508915902 +0000 UTC" Jun 07 09:08:12.271304 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:12.271301 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13808h38m3.237619649s" Jun 07 09:08:12.294100 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:12.294063 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:12.294262 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:12.294206 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:12.294262 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:12.294075 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:12.294375 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:12.294320 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:12.294375 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:12.294063 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:12.294477 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:12.294394 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:12.332077 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:12.332028 2578 generic.go:356] "Generic (PLEG): container finished" podID="7476c89bb59e907acdd0c6bfd86a930b" containerID="69b22ac7eaa496859853a83b45094172ba2d615e06ca77da02324f094d7ede66" exitCode=0 Jun 07 09:08:12.332984 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:12.332960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" event={"ID":"7476c89bb59e907acdd0c6bfd86a930b","Type":"ContainerDied","Data":"69b22ac7eaa496859853a83b45094172ba2d615e06ca77da02324f094d7ede66"} Jun 07 09:08:13.370334 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:13.370295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" event={"ID":"7476c89bb59e907acdd0c6bfd86a930b","Type":"ContainerStarted","Data":"384eabf89e7ff6360f3371152fae7564b9f10debd97c69a893ba8dfadbdcfad0"} Jun 07 09:08:13.873616 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:13.873565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:13.873805 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:13.873640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:13.873867 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:13.873799 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:13.873867 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:13.873820 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:13.873867 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:13.873833 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zjs77 for pod openshift-network-diagnostics/network-check-target-b7fds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:13.874243 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:13.873891 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77 podName:77afbb38-7007-42a8-8d2e-6203b109a0a0 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:17.873871923 +0000 UTC m=+10.018932837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjs77" (UniqueName: "kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77") pod "network-check-target-b7fds" (UID: "77afbb38-7007-42a8-8d2e-6203b109a0a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:13.874315 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:13.874297 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:13.874362 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:13.874352 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret podName:e38f9e64-890b-4696-99fd-2ca38d35a05d nodeName:}" failed. No retries permitted until 2026-06-07 09:08:17.874336363 +0000 UTC m=+10.019397276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret") pod "global-pull-secret-syncer-n4kfk" (UID: "e38f9e64-890b-4696-99fd-2ca38d35a05d") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:13.974349 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:13.974313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:13.974531 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:13.974454 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:13.974531 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:13.974516 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:17.974499614 +0000 UTC m=+10.119560532 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:14.293590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:14.293553 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:14.293590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:14.293583 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:14.293831 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:14.293553 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:14.293831 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:14.293706 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:14.293831 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:14.293768 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:14.294009 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:14.293835 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:16.293844 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:16.293810 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:16.294350 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:16.293956 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:16.294350 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:16.294343 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:16.294487 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:16.294449 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:16.294609 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:16.294588 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:16.294823 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:16.294725 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:17.909874 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:17.909805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:17.910353 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:17.909933 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:17.910353 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:17.909963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:17.910353 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:17.909996 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret podName:e38f9e64-890b-4696-99fd-2ca38d35a05d nodeName:}" failed. No retries permitted until 2026-06-07 09:08:25.909979277 +0000 UTC m=+18.055040194 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret") pod "global-pull-secret-syncer-n4kfk" (UID: "e38f9e64-890b-4696-99fd-2ca38d35a05d") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:17.910353 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:17.910058 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:17.910353 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:17.910071 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:17.910353 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:17.910081 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zjs77 for pod openshift-network-diagnostics/network-check-target-b7fds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:17.910353 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:17.910121 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77 podName:77afbb38-7007-42a8-8d2e-6203b109a0a0 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:25.910108423 +0000 UTC m=+18.055169337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjs77" (UniqueName: "kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77") pod "network-check-target-b7fds" (UID: "77afbb38-7007-42a8-8d2e-6203b109a0a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:18.011702 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:18.011174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:18.011702 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:18.011319 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:18.011702 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:18.011384 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:26.011363861 +0000 UTC m=+18.156424760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:18.295343 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:18.294902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:18.295343 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:18.295023 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:18.295343 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:18.295035 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:18.295343 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:18.295139 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:18.295343 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:18.295199 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:18.295343 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:18.295280 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:20.293778 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:20.293746 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:20.294286 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:20.293763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:20.294286 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:20.293862 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:20.294286 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:20.293888 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:20.294286 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:20.294000 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:20.294286 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:20.294093 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:22.293355 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:22.293274 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:22.293355 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:22.293332 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:22.293355 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:22.293274 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:22.293885 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:22.293417 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:22.293885 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:22.293564 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:22.293885 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:22.293651 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:24.293644 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:24.293612 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:24.293644 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:24.293630 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:24.294200 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:24.293627 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:24.294200 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:24.293747 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:24.294200 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:24.293885 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:24.294200 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:24.294010 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:25.969407 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:25.969363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:25.969823 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:25.969524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:25.969823 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:25.969528 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:25.969823 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:25.969601 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret podName:e38f9e64-890b-4696-99fd-2ca38d35a05d nodeName:}" failed. No retries permitted until 2026-06-07 09:08:41.969585016 +0000 UTC m=+34.114645917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret") pod "global-pull-secret-syncer-n4kfk" (UID: "e38f9e64-890b-4696-99fd-2ca38d35a05d") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:25.969823 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:25.969660 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:25.969823 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:25.969676 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:25.969823 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:25.969699 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zjs77 for pod openshift-network-diagnostics/network-check-target-b7fds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:25.969823 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:25.969752 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77 podName:77afbb38-7007-42a8-8d2e-6203b109a0a0 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:41.96973625 +0000 UTC m=+34.114797150 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjs77" (UniqueName: "kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77") pod "network-check-target-b7fds" (UID: "77afbb38-7007-42a8-8d2e-6203b109a0a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:26.070044 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:26.069997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:26.070226 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:26.070157 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:26.070294 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:26.070233 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:42.070211722 +0000 UTC m=+34.215272634 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:26.293192 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:26.293155 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:26.293349 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:26.293155 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:26.293349 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:26.293279 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:26.293349 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:26.293292 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:26.293513 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:26.293380 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:26.293513 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:26.293473 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:28.294241 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:28.294209 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:28.294676 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:28.294323 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:28.294676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:28.294402 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:28.294676 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:28.294516 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:28.294676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:28.294529 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:28.294676 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:28.294631 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:30.294336 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.293957 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:30.295126 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:30.294450 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:30.295126 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.293973 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:30.295126 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:30.294554 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:30.295126 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.293956 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:30.295126 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:30.294623 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:30.405655 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.405616 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" event={"ID":"33c3dd79-440c-41ca-84da-7138d137be32","Type":"ContainerStarted","Data":"0d4adfd9be959a9f4730e360a31955ea11d70f7d391f4b62c5c8889f0377cee2"} Jun 07 09:08:30.407282 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.407251 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n7kht" event={"ID":"dbb55607-63d7-4a8c-b528-e37853d67387","Type":"ContainerStarted","Data":"f28c5f4029765a7a469bd1a249b06007a783d4abae6fe0ee6be6bf417f7ea7a9"} Jun 07 09:08:30.409152 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.409127 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-msxgz" event={"ID":"795d8080-8304-4fe8-822b-723fc3feafbe","Type":"ContainerStarted","Data":"33ed2083281d2e8dde5b38cb6910a8e4c333166eacd3c35dcca8dd2c883a02b8"} Jun 07 09:08:30.412407 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.412357 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:08:30.412723 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.412701 2578 generic.go:356] "Generic (PLEG): container finished" podID="d5cd375f-3dde-4e82-a91e-fe1b006c0cc9" containerID="da04771ca901b67d0dbf0a6a075e430770ef1bfed785e93ef43c40705a4eb8a1" exitCode=1 Jun 07 09:08:30.412817 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.412772 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"432c0879c2a09c5eea7bb1f48eafc6549d1a91b59516a8acea4fde51d4bd7513"} Jun 07 09:08:30.412897 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.412880 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"3f1a642fb07c010a98d23ee64774d1f1dc37fd28998a7f5aac1c1ff6872f225d"} Jun 07 09:08:30.412963 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.412945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"010a900803431619910aaca8eced2a3dc0915b7ac6107523f03c3ff187696d72"} Jun 07 09:08:30.413054 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.413036 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"2d8ba5b6a055a900f008466cc0e4df50e125499ecad00f951f8ca577f487e086"} Jun 07 09:08:30.413113 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.413094 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerDied","Data":"da04771ca901b67d0dbf0a6a075e430770ef1bfed785e93ef43c40705a4eb8a1"} Jun 07 09:08:30.413168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.413136 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"4c2b1fcb963c88d7cf4087afd57c4a26788c31ca8e912dad1acdebe00119c13b"} Jun 07 09:08:30.414344 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.414318 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cv9fw" event={"ID":"e47627b3-c7bf-4e90-9ee8-69919c6b8c13","Type":"ContainerStarted","Data":"7d0a80b7d9b2e432aae49f71cdcd61fd60011e49012294b099b6ab55c56133c8"} Jun 07 09:08:30.415894 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.415868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dfktq" event={"ID":"d7f74ed1-5a63-4261-b482-e4a9e2b2e47b","Type":"ContainerStarted","Data":"dedb5be9330b216add9507e14ed50cb239abb35646ac784b39c9d123a774b13b"} Jun 07 09:08:30.417367 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.417345 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xbns8" event={"ID":"7102b2e3-1052-46e4-8f26-e9da44934255","Type":"ContainerStarted","Data":"473ced08d650927e46deb5b50de0c8f421b6d80922f7825246dc3581b63042e7"} Jun 07 09:08:30.418966 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.418943 2578 generic.go:356] "Generic (PLEG): container finished" podID="8904cd6d-8eaf-4852-8672-9e1651cab35f" containerID="23a3cfe44e092e39ba1e28f142e338561c198cb3cb8577e3e5b531ade777cf7b" exitCode=0 Jun 07 09:08:30.419061 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.418975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dvckp" event={"ID":"8904cd6d-8eaf-4852-8672-9e1651cab35f","Type":"ContainerDied","Data":"23a3cfe44e092e39ba1e28f142e338561c198cb3cb8577e3e5b531ade777cf7b"} Jun 07 09:08:30.423230 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.423193 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-203.ec2.internal" podStartSLOduration=21.423182779 podStartE2EDuration="21.423182779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:13.387389108 +0000 UTC m=+5.532450027" watchObservedRunningTime="2026-06-07 09:08:30.423182779 +0000 UTC m=+22.568243698" Jun 07 09:08:30.443080 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.443046 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-n7kht" podStartSLOduration=3.988305027 podStartE2EDuration="22.443034097s" totalImagesPullingTime="18.45472907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:30.426548547 +0000 UTC m=+22.571609465" watchObservedRunningTime="2026-06-07 09:08:30.443034097 +0000 UTC m=+22.588095013" Jun 07 09:08:30.443402 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.443378 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xbns8" podStartSLOduration=8.677488344 podStartE2EDuration="22.443369803s" totalImagesPullingTime="13.765881459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:30.443006959 +0000 UTC m=+22.588067876" watchObservedRunningTime="2026-06-07 09:08:30.443369803 +0000 UTC m=+22.588430758" Jun 07 09:08:30.464611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.464573 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-msxgz" podStartSLOduration=4.010939601 podStartE2EDuration="22.464560602s" totalImagesPullingTime="18.453621001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:30.46441123 +0000 UTC m=+22.609472149" watchObservedRunningTime="2026-06-07 09:08:30.464560602 +0000 UTC m=+22.609621520" Jun 07 09:08:30.482247 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.482203 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dfktq" podStartSLOduration=4.035134878 podStartE2EDuration="22.482193715s" totalImagesPullingTime="18.447058837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:30.481887155 +0000 UTC m=+22.626948076" watchObservedRunningTime="2026-06-07 09:08:30.482193715 +0000 UTC m=+22.627254633" Jun 07 09:08:30.524432 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:30.524392 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cv9fw" podStartSLOduration=4.06411112 podStartE2EDuration="22.52437978s" totalImagesPullingTime="18.46026866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:30.524326475 +0000 UTC m=+22.669387394" watchObservedRunningTime="2026-06-07 09:08:30.52437978 +0000 UTC m=+22.669440700" Jun 07 09:08:31.106753 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:31.106595 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 07 09:08:31.296674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:31.296532 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-07T09:08:31.106747138Z","UUID":"2775ebba-c326-4029-b3fb-9e7fdd6050e9","Handler":null,"Name":"","Endpoint":""} Jun 07 09:08:31.298078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:31.298058 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 07 09:08:31.298242 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:31.298099 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 07 09:08:31.425827 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:31.425785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vqv6d" event={"ID":"ba9326ba-4de3-4c15-91dd-d7eb6fa9b1c4","Type":"ContainerStarted","Data":"021fc7e06cd7ee2b95b74343850b545d21108c7e0d10becd6198ec79dbb8d902"} Jun 07 09:08:31.427996 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:31.427965 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" event={"ID":"33c3dd79-440c-41ca-84da-7138d137be32","Type":"ContainerStarted","Data":"947ea35e0c6496cd7bd0a2da8fc0c393fe630dfa7043544c21a484e1547e3506"} Jun 07 09:08:31.444996 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:31.444953 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vqv6d" podStartSLOduration=5.000595753 podStartE2EDuration="23.4449194s" totalImagesPullingTime="18.444323647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:31.444244153 +0000 UTC m=+23.589305072" watchObservedRunningTime="2026-06-07 09:08:31.4449194 +0000 UTC m=+23.589980321" Jun 07 09:08:32.296544 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.296517 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:32.296671 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.296617 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:32.296671 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:32.296627 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:32.296671 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.296644 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:32.297132 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:32.296747 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:32.297132 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:32.296817 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:32.396808 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.396784 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:32.397346 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.397325 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:32.432553 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.432525 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" event={"ID":"33c3dd79-440c-41ca-84da-7138d137be32","Type":"ContainerStarted","Data":"656641300e14272e67ae49f889173ead940f307bc6faa8e7b5d5522822a1a675"} Jun 07 09:08:32.432770 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.432751 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:32.433189 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.433164 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xbns8" Jun 07 09:08:32.465847 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:32.465763 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnl9d" podStartSLOduration=12.098502098 podStartE2EDuration="24.465743755s" totalImagesPullingTime="12.367241657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:32.449342097 +0000 UTC m=+24.594403019" watchObservedRunningTime="2026-06-07 09:08:32.465743755 +0000 UTC m=+24.610804675" Jun 07 09:08:33.437514 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:33.437489 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:08:33.438204 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:33.438117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"ff9bafbc712da61dbac1624fefafc585a67829914363f032ee675ac5cdd3e616"} Jun 07 09:08:34.293857 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:34.293828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:34.294049 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:34.293828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:34.294049 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:34.293984 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:34.294174 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:34.294151 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:34.294420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:34.294020 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:34.294499 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:34.294483 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:35.443361 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:35.443178 2578 generic.go:356] "Generic (PLEG): container finished" podID="8904cd6d-8eaf-4852-8672-9e1651cab35f" containerID="810698cc0b129e0b4d55b20b4bfa32979d37acb1a71417b1b6af14195eb942b3" exitCode=0 Jun 07 09:08:35.443935 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:35.443220 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dvckp" event={"ID":"8904cd6d-8eaf-4852-8672-9e1651cab35f","Type":"ContainerDied","Data":"810698cc0b129e0b4d55b20b4bfa32979d37acb1a71417b1b6af14195eb942b3"} Jun 07 09:08:35.446979 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:35.446961 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:08:35.447292 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:35.447274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"15f0a21580ef303bfe4e65006207af8208cb56126b418ee232384bcc5692f8a8"} Jun 07 09:08:35.447536 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:35.447517 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:35.447632 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:35.447569 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:35.447722 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:35.447707 2578 scope.go:117] "RemoveContainer" containerID="da04771ca901b67d0dbf0a6a075e430770ef1bfed785e93ef43c40705a4eb8a1" Jun 07 09:08:35.463844 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:35.463824 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:36.293352 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.293176 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:36.293485 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.293224 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:36.293485 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:36.293427 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:36.293547 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:36.293502 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:36.293547 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.293253 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:36.293613 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:36.293579 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:36.451733 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.451635 2578 generic.go:356] "Generic (PLEG): container finished" podID="8904cd6d-8eaf-4852-8672-9e1651cab35f" containerID="11cb955908ae873798970248bb12d56767b132021be4954fe22e1ed778a38bc9" exitCode=0 Jun 07 09:08:36.451733 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.451724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dvckp" event={"ID":"8904cd6d-8eaf-4852-8672-9e1651cab35f","Type":"ContainerDied","Data":"11cb955908ae873798970248bb12d56767b132021be4954fe22e1ed778a38bc9"} Jun 07 09:08:36.455598 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.455583 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:08:36.455962 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.455944 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" event={"ID":"d5cd375f-3dde-4e82-a91e-fe1b006c0cc9","Type":"ContainerStarted","Data":"a542543d69aceb47bcf64da3de35d7b84b6f194bbcf795ecf5f66b216a74fb0d"} Jun 07 09:08:36.456210 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.456182 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:36.471549 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.471529 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:08:36.498810 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.498772 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" podStartSLOduration=9.984401126 podStartE2EDuration="28.498758665s" totalImagesPullingTime="18.514357539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:36.498341152 +0000 UTC m=+28.643402212" watchObservedRunningTime="2026-06-07 09:08:36.498758665 +0000 UTC m=+28.643819583" Jun 07 09:08:36.500023 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.499998 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4kfk"] Jun 07 09:08:36.500126 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.500095 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:36.500174 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:36.500165 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:36.502980 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.502956 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p4shp"] Jun 07 09:08:36.503080 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.503057 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:36.503176 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:36.503157 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:36.503727 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.503709 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b7fds"] Jun 07 09:08:36.503819 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:36.503801 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:36.503916 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:36.503891 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:37.460539 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:37.460452 2578 generic.go:356] "Generic (PLEG): container finished" podID="8904cd6d-8eaf-4852-8672-9e1651cab35f" containerID="b76eeb10766cd7331f32f1a746784a6b30346a6df92bea47383a670a92986e29" exitCode=0 Jun 07 09:08:37.460539 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:37.460516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dvckp" event={"ID":"8904cd6d-8eaf-4852-8672-9e1651cab35f","Type":"ContainerDied","Data":"b76eeb10766cd7331f32f1a746784a6b30346a6df92bea47383a670a92986e29"} Jun 07 09:08:38.298254 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:38.297814 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:38.298254 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:38.297910 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:38.298254 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:38.298004 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:38.298254 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:38.298097 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:38.298254 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:38.298126 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:38.298254 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:38.298187 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:40.293183 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:40.293146 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:40.293883 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:40.293145 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:40.293883 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:40.293278 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:40.293883 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:40.293352 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:40.293883 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:40.293415 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:40.293883 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:40.293347 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:41.994611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:41.994583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:41.995104 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:41.994634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:41.995104 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:41.994744 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:41.995104 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:41.994747 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:41.995104 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:41.994761 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:41.995104 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:41.994776 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zjs77 for pod openshift-network-diagnostics/network-check-target-b7fds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:41.995104 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:41.994827 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret podName:e38f9e64-890b-4696-99fd-2ca38d35a05d nodeName:}" failed. No retries permitted until 2026-06-07 09:09:13.994806583 +0000 UTC m=+66.139867491 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret") pod "global-pull-secret-syncer-n4kfk" (UID: "e38f9e64-890b-4696-99fd-2ca38d35a05d") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:41.995104 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:41.994845 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77 podName:77afbb38-7007-42a8-8d2e-6203b109a0a0 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:13.994836897 +0000 UTC m=+66.139897799 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zjs77" (UniqueName: "kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77") pod "network-check-target-b7fds" (UID: "77afbb38-7007-42a8-8d2e-6203b109a0a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:42.095453 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.095382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:42.095588 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:42.095522 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:42.095588 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:42.095578 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:14.095564184 +0000 UTC m=+66.240625081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:42.293146 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.292900 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:42.293257 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.292910 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:42.293299 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:42.293255 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4kfk" podUID="e38f9e64-890b-4696-99fd-2ca38d35a05d" Jun 07 09:08:42.293387 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:42.293363 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b7fds" podUID="77afbb38-7007-42a8-8d2e-6203b109a0a0" Jun 07 09:08:42.293387 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.292963 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:42.293506 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:42.293487 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:08:42.475440 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.475359 2578 generic.go:356] "Generic (PLEG): container finished" podID="8904cd6d-8eaf-4852-8672-9e1651cab35f" containerID="501df612d0bb280ab53362560b2c4477b63deaf42ac3d6a7a78948951c54d88e" exitCode=0 Jun 07 09:08:42.475440 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.475416 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dvckp" event={"ID":"8904cd6d-8eaf-4852-8672-9e1651cab35f","Type":"ContainerDied","Data":"501df612d0bb280ab53362560b2c4477b63deaf42ac3d6a7a78948951c54d88e"} Jun 07 09:08:42.669442 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.669422 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-203.ec2.internal" event="NodeReady" Jun 07 09:08:42.669564 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.669554 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 07 09:08:42.702468 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.702442 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c97d5c65c-w8rrw"] Jun 07 09:08:42.711278 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.711261 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.713283 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.713260 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gmnbx"] Jun 07 09:08:42.713989 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.713964 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 07 09:08:42.714076 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.714045 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 07 09:08:42.714138 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.714122 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 07 09:08:42.714396 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.714381 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sqrsh\"" Jun 07 09:08:42.719560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.719539 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 07 09:08:42.723143 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.723127 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c97d5c65c-w8rrw"] Jun 07 09:08:42.723237 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.723153 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gqbxw"] Jun 07 09:08:42.723304 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.723244 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gmnbx" Jun 07 09:08:42.726028 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.725898 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 07 09:08:42.726447 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.726426 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gpvh7\"" Jun 07 09:08:42.726529 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.726465 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 07 09:08:42.726529 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.726472 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 07 09:08:42.735368 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.735346 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gmnbx"] Jun 07 09:08:42.735445 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.735392 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gqbxw"] Jun 07 09:08:42.735504 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.735493 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:42.738515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.738495 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmvt5\"" Jun 07 09:08:42.738611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.738559 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 07 09:08:42.738611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.738573 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 07 09:08:42.802407 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802384 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phdfx\" (UniqueName: \"kubernetes.io/projected/dc36ff62-339f-4ea0-92fb-3f15cfebdd00-kube-api-access-phdfx\") pod \"ingress-canary-gmnbx\" (UID: \"dc36ff62-339f-4ea0-92fb-3f15cfebdd00\") " pod="openshift-ingress-canary/ingress-canary-gmnbx" Jun 07 09:08:42.802565 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-image-registry-private-configuration\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.802565 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802439 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc36ff62-339f-4ea0-92fb-3f15cfebdd00-cert\") pod \"ingress-canary-gmnbx\" (UID: \"dc36ff62-339f-4ea0-92fb-3f15cfebdd00\") " pod="openshift-ingress-canary/ingress-canary-gmnbx" Jun 07 09:08:42.802565 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802454 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4rrk\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-kube-api-access-l4rrk\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.802565 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-installation-pull-secrets\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.802706 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.802706 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802591 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-bound-sa-token\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.802706 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c21300a-c19c-4af6-aaa0-f4927661a250-ca-trust-extracted\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.802706 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-certificates\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.802706 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.802659 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-trusted-ca\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.903790 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.903762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa52f142-0b4a-40a4-a7e3-3445e168731a-tmp-dir\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:42.903995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.903811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.903995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.903869 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-bound-sa-token\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.903995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.903907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c21300a-c19c-4af6-aaa0-f4927661a250-ca-trust-extracted\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.903995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.903952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:42.903995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.903982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-certificates\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.904241 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:42.904004 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:08:42.904241 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:42.904020 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:08:42.904241 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:42.904080 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:43.404056618 +0000 UTC m=+35.549117530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:08:42.904241 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904007 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-trusted-ca\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.904241 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hww4x\" (UniqueName: \"kubernetes.io/projected/fa52f142-0b4a-40a4-a7e3-3445e168731a-kube-api-access-hww4x\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:42.904241 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phdfx\" (UniqueName: \"kubernetes.io/projected/dc36ff62-339f-4ea0-92fb-3f15cfebdd00-kube-api-access-phdfx\") pod \"ingress-canary-gmnbx\" (UID: \"dc36ff62-339f-4ea0-92fb-3f15cfebdd00\") " pod="openshift-ingress-canary/ingress-canary-gmnbx" Jun 07 09:08:42.904241 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904213 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-image-registry-private-configuration\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.904609 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904254 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc36ff62-339f-4ea0-92fb-3f15cfebdd00-cert\") pod \"ingress-canary-gmnbx\" (UID: \"dc36ff62-339f-4ea0-92fb-3f15cfebdd00\") " pod="openshift-ingress-canary/ingress-canary-gmnbx" Jun 07 09:08:42.904609 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4rrk\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-kube-api-access-l4rrk\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.904609 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa52f142-0b4a-40a4-a7e3-3445e168731a-config-volume\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:42.904609 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904332 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c21300a-c19c-4af6-aaa0-f4927661a250-ca-trust-extracted\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.904609 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-installation-pull-secrets\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.904826 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904615 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-certificates\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.904940 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.904901 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-trusted-ca\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.908275 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.908247 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-installation-pull-secrets\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.908405 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.908249 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-image-registry-private-configuration\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.908405 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.908282 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc36ff62-339f-4ea0-92fb-3f15cfebdd00-cert\") pod \"ingress-canary-gmnbx\" (UID: \"dc36ff62-339f-4ea0-92fb-3f15cfebdd00\") " pod="openshift-ingress-canary/ingress-canary-gmnbx" Jun 07 09:08:42.915274 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.915249 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4rrk\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-kube-api-access-l4rrk\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.915794 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.915776 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-bound-sa-token\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:42.916009 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:42.915990 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phdfx\" (UniqueName: \"kubernetes.io/projected/dc36ff62-339f-4ea0-92fb-3f15cfebdd00-kube-api-access-phdfx\") pod \"ingress-canary-gmnbx\" (UID: \"dc36ff62-339f-4ea0-92fb-3f15cfebdd00\") " pod="openshift-ingress-canary/ingress-canary-gmnbx" Jun 07 09:08:43.005187 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.005121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:43.005187 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.005160 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hww4x\" (UniqueName: \"kubernetes.io/projected/fa52f142-0b4a-40a4-a7e3-3445e168731a-kube-api-access-hww4x\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:43.005577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.005194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa52f142-0b4a-40a4-a7e3-3445e168731a-config-volume\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:43.005577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.005223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa52f142-0b4a-40a4-a7e3-3445e168731a-tmp-dir\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:43.005577 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:43.005269 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:08:43.005577 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:43.005329 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:08:43.505313748 +0000 UTC m=+35.650374645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:08:43.005577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.005555 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa52f142-0b4a-40a4-a7e3-3445e168731a-tmp-dir\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:43.005788 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.005745 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa52f142-0b4a-40a4-a7e3-3445e168731a-config-volume\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:43.013701 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.013665 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hww4x\" (UniqueName: \"kubernetes.io/projected/fa52f142-0b4a-40a4-a7e3-3445e168731a-kube-api-access-hww4x\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:43.032856 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.032833 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gmnbx" Jun 07 09:08:43.197642 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.197616 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gmnbx"] Jun 07 09:08:43.200536 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:08:43.200503 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc36ff62_339f_4ea0_92fb_3f15cfebdd00.slice/crio-f6e640701678b34e2172504304f5c163ec2ea0a516c9cb5f2050add26c7d175d WatchSource:0}: Error finding container f6e640701678b34e2172504304f5c163ec2ea0a516c9cb5f2050add26c7d175d: Status 404 returned error can't find the container with id f6e640701678b34e2172504304f5c163ec2ea0a516c9cb5f2050add26c7d175d Jun 07 09:08:43.408022 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.407940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:43.408172 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:43.408093 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:08:43.408172 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:43.408113 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:08:43.408172 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:43.408173 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:44.40815437 +0000 UTC m=+36.553215268 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:08:43.479143 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.479110 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gmnbx" event={"ID":"dc36ff62-339f-4ea0-92fb-3f15cfebdd00","Type":"ContainerStarted","Data":"f6e640701678b34e2172504304f5c163ec2ea0a516c9cb5f2050add26c7d175d"} Jun 07 09:08:43.482083 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.482060 2578 generic.go:356] "Generic (PLEG): container finished" podID="8904cd6d-8eaf-4852-8672-9e1651cab35f" containerID="6af11abf6581a997c5dd5f06b7696b1879af27222faca6b1a37821ba9173331b" exitCode=0 Jun 07 09:08:43.482191 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.482108 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dvckp" event={"ID":"8904cd6d-8eaf-4852-8672-9e1651cab35f","Type":"ContainerDied","Data":"6af11abf6581a997c5dd5f06b7696b1879af27222faca6b1a37821ba9173331b"} Jun 07 09:08:43.508711 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:43.508686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:43.508828 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:43.508800 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:08:43.508890 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:43.508876 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:08:44.508856632 +0000 UTC m=+36.653917544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:08:44.293367 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.293168 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:08:44.293983 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.293167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:08:44.293983 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.293263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:08:44.297544 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.297522 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 07 09:08:44.297691 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.297572 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-99ths\"" Jun 07 09:08:44.297691 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.297527 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 07 09:08:44.297821 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.297738 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jh2fl\"" Jun 07 09:08:44.297821 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.297743 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 07 09:08:44.297821 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.297779 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 07 09:08:44.417130 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.417097 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:44.417297 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:44.417267 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:08:44.417297 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:44.417287 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:08:44.417409 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:44.417345 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:46.417327305 +0000 UTC m=+38.562388208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:08:44.487834 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.487802 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dvckp" event={"ID":"8904cd6d-8eaf-4852-8672-9e1651cab35f","Type":"ContainerStarted","Data":"ba6622b23275187dbc5c091c59ad69221a8fdd811ae0c4bcd167f0185569d941"} Jun 07 09:08:44.511786 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.511721 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dvckp" podStartSLOduration=8.609082963 podStartE2EDuration="36.511702526s" totalImagesPullingTime="27.902619563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:44.510337962 +0000 UTC m=+36.655398881" watchObservedRunningTime="2026-06-07 09:08:44.511702526 +0000 UTC m=+36.656763442" Jun 07 09:08:44.517794 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:44.517771 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:44.517944 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:44.517911 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:08:44.518009 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:44.517986 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:08:46.517972588 +0000 UTC m=+38.663033485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:08:46.434986 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:46.434893 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:46.435559 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:46.435056 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:08:46.435559 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:46.435076 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:08:46.435559 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:46.435131 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:50.435115543 +0000 UTC m=+42.580176440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:08:46.494490 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:46.494457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gmnbx" event={"ID":"dc36ff62-339f-4ea0-92fb-3f15cfebdd00","Type":"ContainerStarted","Data":"8d51bcfd2b9875b59008a843f6c9209023b099b2494030eb0b6c7f29f38ab7f4"} Jun 07 09:08:46.511109 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:46.511063 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gmnbx" podStartSLOduration=2.033886233 podStartE2EDuration="4.511046506s" totalImagesPullingTime="2.477160273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:46.510964828 +0000 UTC m=+38.656025745" watchObservedRunningTime="2026-06-07 09:08:46.511046506 +0000 UTC m=+38.656107426" Jun 07 09:08:46.536206 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:46.536180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:46.536354 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:46.536336 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:08:46.536415 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:46.536406 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:08:50.536386372 +0000 UTC m=+42.681447271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:08:50.463560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:50.463363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:50.464068 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:50.463533 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:08:50.464068 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:50.463638 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:08:50.464068 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:50.463698 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:58.463681893 +0000 UTC m=+50.608742791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:08:50.564855 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:50.564813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:50.565027 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:50.564987 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:08:50.565084 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:50.565048 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:08:58.565030934 +0000 UTC m=+50.710091842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:08:58.520323 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:58.520292 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:08:58.520684 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:58.520395 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:08:58.520684 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:58.520406 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:08:58.520684 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:58.520452 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:14.520438848 +0000 UTC m=+66.665499744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:08:58.621130 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:08:58.621093 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:08:58.621320 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:58.621246 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:08:58.621320 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:08:58.621319 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:09:14.621299482 +0000 UTC m=+66.766360378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:09:08.473023 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:08.472994 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6tljf" Jun 07 09:09:14.036063 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.036023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:09:14.036464 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.036091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:09:14.039112 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.039084 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 07 09:09:14.039235 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.039158 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 07 09:09:14.049045 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.049023 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 07 09:09:14.049815 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.049797 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38f9e64-890b-4696-99fd-2ca38d35a05d-original-pull-secret\") pod \"global-pull-secret-syncer-n4kfk\" (UID: \"e38f9e64-890b-4696-99fd-2ca38d35a05d\") " pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:09:14.059149 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.059133 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjs77\" (UniqueName: \"kubernetes.io/projected/77afbb38-7007-42a8-8d2e-6203b109a0a0-kube-api-access-zjs77\") pod \"network-check-target-b7fds\" (UID: \"77afbb38-7007-42a8-8d2e-6203b109a0a0\") " pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:09:14.136899 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.136863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:09:14.140073 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.140054 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 07 09:09:14.147408 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:14.147394 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 07 09:09:14.147487 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:14.147450 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:10:18.147434017 +0000 UTC m=+130.292494914 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : secret "metrics-daemon-secret" not found Jun 07 09:09:14.308582 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.308503 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-99ths\"" Jun 07 09:09:14.312769 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.312749 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4kfk" Jun 07 09:09:14.315886 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.315872 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:09:14.500952 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.500903 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4kfk"] Jun 07 09:09:14.503863 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:09:14.503834 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38f9e64_890b_4696_99fd_2ca38d35a05d.slice/crio-bac78c246da334da501b03627b42f24d603a0a5d2187a5b7ed1b73269d985e32 WatchSource:0}: Error finding container bac78c246da334da501b03627b42f24d603a0a5d2187a5b7ed1b73269d985e32: Status 404 returned error can't find the container with id bac78c246da334da501b03627b42f24d603a0a5d2187a5b7ed1b73269d985e32 Jun 07 09:09:14.515870 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.515851 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b7fds"] Jun 07 09:09:14.519309 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:09:14.519280 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77afbb38_7007_42a8_8d2e_6203b109a0a0.slice/crio-5f310293cd07bd972d3e3e55c931fce9dbd14e966850e9b20d9e7a23969321b1 WatchSource:0}: Error finding container 5f310293cd07bd972d3e3e55c931fce9dbd14e966850e9b20d9e7a23969321b1: Status 404 returned error can't find the container with id 5f310293cd07bd972d3e3e55c931fce9dbd14e966850e9b20d9e7a23969321b1 Jun 07 09:09:14.544176 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.544145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:09:14.544279 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:14.544267 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:09:14.544321 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:14.544282 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:09:14.544363 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:14.544336 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:46.544319884 +0000 UTC m=+98.689380800 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:09:14.559605 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.559542 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b7fds" event={"ID":"77afbb38-7007-42a8-8d2e-6203b109a0a0","Type":"ContainerStarted","Data":"5f310293cd07bd972d3e3e55c931fce9dbd14e966850e9b20d9e7a23969321b1"} Jun 07 09:09:14.560513 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.560493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4kfk" event={"ID":"e38f9e64-890b-4696-99fd-2ca38d35a05d","Type":"ContainerStarted","Data":"bac78c246da334da501b03627b42f24d603a0a5d2187a5b7ed1b73269d985e32"} Jun 07 09:09:14.644827 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:14.644803 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:09:14.644919 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:14.644900 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:14.644970 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:14.644961 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:09:46.644947506 +0000 UTC m=+98.790008406 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:09:19.573986 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:19.573949 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b7fds" event={"ID":"77afbb38-7007-42a8-8d2e-6203b109a0a0","Type":"ContainerStarted","Data":"98edac8c79836dea400965d6b5d3a720231adc6c721bda7a561001cf0fc86cd8"} Jun 07 09:09:19.574480 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:19.574040 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:09:19.575383 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:19.575364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4kfk" event={"ID":"e38f9e64-890b-4696-99fd-2ca38d35a05d","Type":"ContainerStarted","Data":"30dfaffb0f604d868972305e5dbf9cdfa085ba2123f9a6e2d07e3ae1c03b08fd"} Jun 07 09:09:19.591640 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:19.591599 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b7fds" podStartSLOduration=67.010822966 podStartE2EDuration="1m11.591588031s" totalImagesPullingTime="4.580765065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:09:19.59044645 +0000 UTC m=+71.735507366" watchObservedRunningTime="2026-06-07 09:09:19.591588031 +0000 UTC m=+71.736648948" Jun 07 09:09:19.605504 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:19.605468 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n4kfk" podStartSLOduration=67.006962866 podStartE2EDuration="1m11.605455566s" totalImagesPullingTime="4.5984927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:09:19.605166255 +0000 UTC m=+71.750227166" watchObservedRunningTime="2026-06-07 09:09:19.605455566 +0000 UTC m=+71.750516478" Jun 07 09:09:46.579229 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:46.579193 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:09:46.579652 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:46.579359 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:09:46.579652 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:46.579383 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:09:46.579652 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:46.579455 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:10:50.579431676 +0000 UTC m=+162.724492589 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:09:46.680466 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:46.680425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:09:46.680610 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:46.680573 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:46.680644 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:09:46.680626 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:10:50.680612318 +0000 UTC m=+162.825673218 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:09:50.579877 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:09:50.579845 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b7fds" Jun 07 09:10:18.202957 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:10:18.202909 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:10:18.203437 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:18.203028 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 07 09:10:18.203437 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:18.203086 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs podName:dfbdde5e-d359-422e-bf75-55f3221bd8f3 nodeName:}" failed. No retries permitted until 2026-06-07 09:12:20.203071081 +0000 UTC m=+252.348131979 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs") pod "network-metrics-daemon-p4shp" (UID: "dfbdde5e-d359-422e-bf75-55f3221bd8f3") : secret "metrics-daemon-secret" not found Jun 07 09:10:45.721521 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:45.721476 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" podUID="3c21300a-c19c-4af6-aaa0-f4927661a250" Jun 07 09:10:45.743853 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:45.743817 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gqbxw" podUID="fa52f142-0b4a-40a4-a7e3-3445e168731a" Jun 07 09:10:45.773733 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:10:45.773707 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:10:47.318850 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:47.318793 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-p4shp" podUID="dfbdde5e-d359-422e-bf75-55f3221bd8f3" Jun 07 09:10:50.624397 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:10:50.624334 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:10:50.624817 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:50.624483 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:10:50.624817 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:50.624498 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c97d5c65c-w8rrw: secret "image-registry-tls" not found Jun 07 09:10:50.624817 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:50.624550 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls podName:3c21300a-c19c-4af6-aaa0-f4927661a250 nodeName:}" failed. No retries permitted until 2026-06-07 09:12:52.6245355 +0000 UTC m=+284.769596397 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls") pod "image-registry-6c97d5c65c-w8rrw" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250") : secret "image-registry-tls" not found Jun 07 09:10:50.725046 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:10:50.725008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:10:50.725225 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:50.725154 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:10:50.725267 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:10:50.725230 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls podName:fa52f142-0b4a-40a4-a7e3-3445e168731a nodeName:}" failed. No retries permitted until 2026-06-07 09:12:52.725214851 +0000 UTC m=+284.870275748 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls") pod "dns-default-gqbxw" (UID: "fa52f142-0b4a-40a4-a7e3-3445e168731a") : secret "dns-default-metrics-tls" not found Jun 07 09:10:57.293080 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:10:57.292984 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gqbxw" Jun 07 09:10:58.293994 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:10:58.293965 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:11:27.131557 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.131526 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv"] Jun 07 09:11:27.133552 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.133537 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.135263 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.135236 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk"] Jun 07 09:11:27.136630 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.136612 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 07 09:11:27.136844 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.136830 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:11:27.136844 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.136837 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 07 09:11:27.137012 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.136995 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42"] Jun 07 09:11:27.137120 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.137104 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.137381 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.137341 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 07 09:11:27.137751 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.137734 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-l7srd\"" Jun 07 09:11:27.138725 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.138712 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-896549976-vf2x8"] Jun 07 09:11:27.138832 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.138817 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.139631 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.139615 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 07 09:11:27.139995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.139978 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:11:27.140091 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.139987 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 07 09:11:27.140386 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.140370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.140785 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.140767 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 07 09:11:27.141574 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.141345 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ww7nq\"" Jun 07 09:11:27.141641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.141610 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 07 09:11:27.141758 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.141736 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 07 09:11:27.141877 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.141823 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 07 09:11:27.143387 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.143369 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 07 09:11:27.143489 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.143448 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tsmnx\"" Jun 07 09:11:27.143938 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.143882 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 07 09:11:27.144041 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.144026 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 07 09:11:27.144107 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.144094 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 07 09:11:27.144171 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.144113 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 07 09:11:27.144411 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.144398 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rq7p7\"" Jun 07 09:11:27.151340 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.151323 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk"] Jun 07 09:11:27.159167 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.159144 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 07 09:11:27.160284 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.159104 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42"] Jun 07 09:11:27.160504 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.160477 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-vf2x8"] Jun 07 09:11:27.161080 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.161057 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv"] Jun 07 09:11:27.231967 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.231940 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd"] Jun 07 09:11:27.233867 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.233853 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd" Jun 07 09:11:27.237728 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.237709 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 07 09:11:27.237843 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.237748 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cgv8n\"" Jun 07 09:11:27.238676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.238652 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:11:27.249342 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.249320 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd"] Jun 07 09:11:27.271109 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55194eb9-d80a-43f9-aa3e-a1355156a565-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.271229 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271138 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61c5adf4-9ecc-4355-a255-2aa644c130b9-serving-cert\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.271229 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61c5adf4-9ecc-4355-a255-2aa644c130b9-service-ca-bundle\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.271229 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271209 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh4xx\" (UniqueName: \"kubernetes.io/projected/55194eb9-d80a-43f9-aa3e-a1355156a565-kube-api-access-wh4xx\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.271361 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271264 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.271361 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61c5adf4-9ecc-4355-a255-2aa644c130b9-tmp\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.271361 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.271361 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61c5adf4-9ecc-4355-a255-2aa644c130b9-snapshots\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.271517 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.271517 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh5rw\" (UniqueName: \"kubernetes.io/projected/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-kube-api-access-jh5rw\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.271517 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.271517 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271512 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61c5adf4-9ecc-4355-a255-2aa644c130b9-trusted-ca-bundle\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.271652 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55194eb9-d80a-43f9-aa3e-a1355156a565-config\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.271652 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271551 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxbnz\" (UniqueName: \"kubernetes.io/projected/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-kube-api-access-qxbnz\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.271652 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.271576 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l28v9\" (UniqueName: \"kubernetes.io/projected/61c5adf4-9ecc-4355-a255-2aa644c130b9-kube-api-access-l28v9\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.330121 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.330098 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7"] Jun 07 09:11:27.331967 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.331954 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7" Jun 07 09:11:27.334606 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.334589 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bj2pq\"" Jun 07 09:11:27.342643 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.342622 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7"] Jun 07 09:11:27.372105 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.372199 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61c5adf4-9ecc-4355-a255-2aa644c130b9-tmp\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.372199 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.372274 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61c5adf4-9ecc-4355-a255-2aa644c130b9-snapshots\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.372313 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.372360 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh5rw\" (UniqueName: \"kubernetes.io/projected/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-kube-api-access-jh5rw\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.372360 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.372460 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372376 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61c5adf4-9ecc-4355-a255-2aa644c130b9-trusted-ca-bundle\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.372460 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372405 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55194eb9-d80a-43f9-aa3e-a1355156a565-config\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.372460 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxbnz\" (UniqueName: \"kubernetes.io/projected/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-kube-api-access-qxbnz\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.372607 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l28v9\" (UniqueName: \"kubernetes.io/projected/61c5adf4-9ecc-4355-a255-2aa644c130b9-kube-api-access-l28v9\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.372607 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372491 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nmx9\" (UniqueName: \"kubernetes.io/projected/a58d20d0-9c6c-48d6-b3f8-52cd282b086f-kube-api-access-5nmx9\") pod \"volume-data-source-validator-78dd696465-fcrnd\" (UID: \"a58d20d0-9c6c-48d6-b3f8-52cd282b086f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd" Jun 07 09:11:27.372607 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:27.372527 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:27.372748 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:27.372619 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls podName:a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:27.87258413 +0000 UTC m=+200.017645029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kkw42" (UID: "a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4") : secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372528 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55194eb9-d80a-43f9-aa3e-a1355156a565-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372863 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372883 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61c5adf4-9ecc-4355-a255-2aa644c130b9-serving-cert\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372917 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61c5adf4-9ecc-4355-a255-2aa644c130b9-service-ca-bundle\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372972 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh4xx\" (UniqueName: \"kubernetes.io/projected/55194eb9-d80a-43f9-aa3e-a1355156a565-kube-api-access-wh4xx\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61c5adf4-9ecc-4355-a255-2aa644c130b9-tmp\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.372887 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/61c5adf4-9ecc-4355-a255-2aa644c130b9-snapshots\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.373161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.373500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.373469 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55194eb9-d80a-43f9-aa3e-a1355156a565-config\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.373907 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.373602 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61c5adf4-9ecc-4355-a255-2aa644c130b9-service-ca-bundle\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.373907 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.373738 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61c5adf4-9ecc-4355-a255-2aa644c130b9-trusted-ca-bundle\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.374454 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.374433 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.374970 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.374955 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55194eb9-d80a-43f9-aa3e-a1355156a565-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.375413 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.375389 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61c5adf4-9ecc-4355-a255-2aa644c130b9-serving-cert\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.381897 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.381826 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxbnz\" (UniqueName: \"kubernetes.io/projected/da36a86e-4fab-4dad-85e3-0c9232d4a5f0-kube-api-access-qxbnz\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-qw7sk\" (UID: \"da36a86e-4fab-4dad-85e3-0c9232d4a5f0\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.382185 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.382167 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28v9\" (UniqueName: \"kubernetes.io/projected/61c5adf4-9ecc-4355-a255-2aa644c130b9-kube-api-access-l28v9\") pod \"insights-operator-896549976-vf2x8\" (UID: \"61c5adf4-9ecc-4355-a255-2aa644c130b9\") " pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.382255 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.382236 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh5rw\" (UniqueName: \"kubernetes.io/projected/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-kube-api-access-jh5rw\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.382293 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.382260 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh4xx\" (UniqueName: \"kubernetes.io/projected/55194eb9-d80a-43f9-aa3e-a1355156a565-kube-api-access-wh4xx\") pod \"service-ca-operator-6bc5ff68f5-wrccv\" (UID: \"55194eb9-d80a-43f9-aa3e-a1355156a565\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.444561 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.444536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" Jun 07 09:11:27.451274 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.451258 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" Jun 07 09:11:27.466259 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.466236 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-vf2x8" Jun 07 09:11:27.473324 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.473298 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m58zq\" (UniqueName: \"kubernetes.io/projected/3db5aff8-90c7-4aa2-bb08-3ed2af0ffa61-kube-api-access-m58zq\") pod \"network-check-source-dbd9cf844-9wmd7\" (UID: \"3db5aff8-90c7-4aa2-bb08-3ed2af0ffa61\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7" Jun 07 09:11:27.473444 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.473427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nmx9\" (UniqueName: \"kubernetes.io/projected/a58d20d0-9c6c-48d6-b3f8-52cd282b086f-kube-api-access-5nmx9\") pod \"volume-data-source-validator-78dd696465-fcrnd\" (UID: \"a58d20d0-9c6c-48d6-b3f8-52cd282b086f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd" Jun 07 09:11:27.481889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.481869 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nmx9\" (UniqueName: \"kubernetes.io/projected/a58d20d0-9c6c-48d6-b3f8-52cd282b086f-kube-api-access-5nmx9\") pod \"volume-data-source-validator-78dd696465-fcrnd\" (UID: \"a58d20d0-9c6c-48d6-b3f8-52cd282b086f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd" Jun 07 09:11:27.541906 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.541879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd" Jun 07 09:11:27.574649 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.574600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m58zq\" (UniqueName: \"kubernetes.io/projected/3db5aff8-90c7-4aa2-bb08-3ed2af0ffa61-kube-api-access-m58zq\") pod \"network-check-source-dbd9cf844-9wmd7\" (UID: \"3db5aff8-90c7-4aa2-bb08-3ed2af0ffa61\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7" Jun 07 09:11:27.582995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.582964 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv"] Jun 07 09:11:27.586197 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:27.586157 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55194eb9_d80a_43f9_aa3e_a1355156a565.slice/crio-391fb1c2831aae744810708fcb1efe9c16ef21f5eff894a5a6ea6548b45aecd4 WatchSource:0}: Error finding container 391fb1c2831aae744810708fcb1efe9c16ef21f5eff894a5a6ea6548b45aecd4: Status 404 returned error can't find the container with id 391fb1c2831aae744810708fcb1efe9c16ef21f5eff894a5a6ea6548b45aecd4 Jun 07 09:11:27.586669 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.586647 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m58zq\" (UniqueName: \"kubernetes.io/projected/3db5aff8-90c7-4aa2-bb08-3ed2af0ffa61-kube-api-access-m58zq\") pod \"network-check-source-dbd9cf844-9wmd7\" (UID: \"3db5aff8-90c7-4aa2-bb08-3ed2af0ffa61\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7" Jun 07 09:11:27.603857 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.603518 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk"] Jun 07 09:11:27.604337 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:27.604304 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda36a86e_4fab_4dad_85e3_0c9232d4a5f0.slice/crio-4dbd155b59b3ffcc0cca2c8c9a0bd2b3bc113f08a0b987b7d0b84b1559893210 WatchSource:0}: Error finding container 4dbd155b59b3ffcc0cca2c8c9a0bd2b3bc113f08a0b987b7d0b84b1559893210: Status 404 returned error can't find the container with id 4dbd155b59b3ffcc0cca2c8c9a0bd2b3bc113f08a0b987b7d0b84b1559893210 Jun 07 09:11:27.617980 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.617956 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-vf2x8"] Jun 07 09:11:27.621249 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:27.621212 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c5adf4_9ecc_4355_a255_2aa644c130b9.slice/crio-b22153f0c2e0590511d5aeb7c96181484a2ab66ccbe085cf4b98d21a6c8d620f WatchSource:0}: Error finding container b22153f0c2e0590511d5aeb7c96181484a2ab66ccbe085cf4b98d21a6c8d620f: Status 404 returned error can't find the container with id b22153f0c2e0590511d5aeb7c96181484a2ab66ccbe085cf4b98d21a6c8d620f Jun 07 09:11:27.640365 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.640297 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7" Jun 07 09:11:27.673159 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.673132 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd"] Jun 07 09:11:27.674099 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:27.674073 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58d20d0_9c6c_48d6_b3f8_52cd282b086f.slice/crio-ab4d1075e220e1cb2bbbcd4eb37a67402cf3c9ec12fbf4baab5eb38f8a7bff7f WatchSource:0}: Error finding container ab4d1075e220e1cb2bbbcd4eb37a67402cf3c9ec12fbf4baab5eb38f8a7bff7f: Status 404 returned error can't find the container with id ab4d1075e220e1cb2bbbcd4eb37a67402cf3c9ec12fbf4baab5eb38f8a7bff7f Jun 07 09:11:27.761502 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.761448 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7"] Jun 07 09:11:27.764756 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:27.764730 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db5aff8_90c7_4aa2_bb08_3ed2af0ffa61.slice/crio-7f0f5710f586ca750fcb47a32a4d3dcb8d9acd02b52c1125bc4f25e6f943556e WatchSource:0}: Error finding container 7f0f5710f586ca750fcb47a32a4d3dcb8d9acd02b52c1125bc4f25e6f943556e: Status 404 returned error can't find the container with id 7f0f5710f586ca750fcb47a32a4d3dcb8d9acd02b52c1125bc4f25e6f943556e Jun 07 09:11:27.870820 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.870790 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7" event={"ID":"3db5aff8-90c7-4aa2-bb08-3ed2af0ffa61","Type":"ContainerStarted","Data":"dccae6deb0a8c887cabd6b1eaab0912665a72e906aad1eee72943f3b3fbf096f"} Jun 07 09:11:27.870993 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.870841 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7" event={"ID":"3db5aff8-90c7-4aa2-bb08-3ed2af0ffa61","Type":"ContainerStarted","Data":"7f0f5710f586ca750fcb47a32a4d3dcb8d9acd02b52c1125bc4f25e6f943556e"} Jun 07 09:11:27.871888 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.871869 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd" event={"ID":"a58d20d0-9c6c-48d6-b3f8-52cd282b086f","Type":"ContainerStarted","Data":"ab4d1075e220e1cb2bbbcd4eb37a67402cf3c9ec12fbf4baab5eb38f8a7bff7f"} Jun 07 09:11:27.873063 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.873041 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-vf2x8" event={"ID":"61c5adf4-9ecc-4355-a255-2aa644c130b9","Type":"ContainerStarted","Data":"b22153f0c2e0590511d5aeb7c96181484a2ab66ccbe085cf4b98d21a6c8d620f"} Jun 07 09:11:27.874141 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.874118 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" event={"ID":"55194eb9-d80a-43f9-aa3e-a1355156a565","Type":"ContainerStarted","Data":"391fb1c2831aae744810708fcb1efe9c16ef21f5eff894a5a6ea6548b45aecd4"} Jun 07 09:11:27.875206 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.875188 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" event={"ID":"da36a86e-4fab-4dad-85e3-0c9232d4a5f0","Type":"ContainerStarted","Data":"4dbd155b59b3ffcc0cca2c8c9a0bd2b3bc113f08a0b987b7d0b84b1559893210"} Jun 07 09:11:27.876475 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.876458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:27.876602 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:27.876589 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:27.876648 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:27.876640 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls podName:a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:28.876626262 +0000 UTC m=+201.021687160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kkw42" (UID: "a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4") : secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:27.887604 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:27.887562 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-9wmd7" podStartSLOduration=0.887545416 podStartE2EDuration="887.545416ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:27.886794196 +0000 UTC m=+200.031855109" watchObservedRunningTime="2026-06-07 09:11:27.887545416 +0000 UTC m=+200.032606318" Jun 07 09:11:28.885399 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:28.884774 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:28.885399 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:28.884959 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:28.885399 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:28.885025 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls podName:a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:30.885003606 +0000 UTC m=+203.030064506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kkw42" (UID: "a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4") : secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:29.883964 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:29.883914 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd" event={"ID":"a58d20d0-9c6c-48d6-b3f8-52cd282b086f","Type":"ContainerStarted","Data":"94336e35f517dae59340ce56a118134bb273f8a348a58163ff5b67a6d3151dda"} Jun 07 09:11:29.900299 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:29.900245 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-fcrnd" podStartSLOduration=1.649732335 podStartE2EDuration="2.900231494s" totalImagesPullingTime="1.250499159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:29.899454388 +0000 UTC m=+202.044515308" watchObservedRunningTime="2026-06-07 09:11:29.900231494 +0000 UTC m=+202.045292391" Jun 07 09:11:30.891823 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:30.891781 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" event={"ID":"da36a86e-4fab-4dad-85e3-0c9232d4a5f0","Type":"ContainerStarted","Data":"9bcba4d47b38869ba8da7edda659a893c4554880c18f60d45b72c5b8a5cd8d4b"} Jun 07 09:11:30.893424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:30.893394 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-vf2x8" event={"ID":"61c5adf4-9ecc-4355-a255-2aa644c130b9","Type":"ContainerStarted","Data":"5ac8ebd62aae91b18a30e56c7fd4c89a0eeaf1800e9f0740bf7201dca320268c"} Jun 07 09:11:30.894901 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:30.894876 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" event={"ID":"55194eb9-d80a-43f9-aa3e-a1355156a565","Type":"ContainerStarted","Data":"fb7b9834d9a812508f68d13a01da38e12b53155a3094031bf4d9babf2ef67fbc"} Jun 07 09:11:30.901970 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:30.901903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:30.902261 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:30.902038 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:30.902261 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:30.902117 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls podName:a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:34.902097811 +0000 UTC m=+207.047158716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kkw42" (UID: "a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4") : secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:30.908237 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:30.908201 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" podStartSLOduration=0.854772088 podStartE2EDuration="3.908187904s" totalImagesPullingTime="3.053415816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:30.907606328 +0000 UTC m=+203.052667273" watchObservedRunningTime="2026-06-07 09:11:30.908187904 +0000 UTC m=+203.053248826" Jun 07 09:11:30.926162 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:30.926112 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" podStartSLOduration=0.849092085 podStartE2EDuration="3.926092309s" totalImagesPullingTime="3.077000224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:30.924592261 +0000 UTC m=+203.069653181" watchObservedRunningTime="2026-06-07 09:11:30.926092309 +0000 UTC m=+203.071153229" Jun 07 09:11:30.941604 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:30.941558 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-896549976-vf2x8" podStartSLOduration=0.899595587 podStartE2EDuration="3.941540048s" totalImagesPullingTime="3.041944461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:30.939719026 +0000 UTC m=+203.084779947" watchObservedRunningTime="2026-06-07 09:11:30.941540048 +0000 UTC m=+203.086600969" Jun 07 09:11:32.398406 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.398368 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk"] Jun 07 09:11:32.400936 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.400908 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" Jun 07 09:11:32.403743 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.403724 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 07 09:11:32.405006 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.404982 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 07 09:11:32.405114 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.405030 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jqpww\"" Jun 07 09:11:32.410408 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.410386 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk"] Jun 07 09:11:32.517008 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.516977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjfb8\" (UniqueName: \"kubernetes.io/projected/ca89a42b-4d21-4d22-9181-e72d02a56659-kube-api-access-vjfb8\") pod \"migrator-955ffdf5f-pqqqk\" (UID: \"ca89a42b-4d21-4d22-9181-e72d02a56659\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" Jun 07 09:11:32.617293 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.617263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjfb8\" (UniqueName: \"kubernetes.io/projected/ca89a42b-4d21-4d22-9181-e72d02a56659-kube-api-access-vjfb8\") pod \"migrator-955ffdf5f-pqqqk\" (UID: \"ca89a42b-4d21-4d22-9181-e72d02a56659\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" Jun 07 09:11:32.625367 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.625327 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjfb8\" (UniqueName: \"kubernetes.io/projected/ca89a42b-4d21-4d22-9181-e72d02a56659-kube-api-access-vjfb8\") pod \"migrator-955ffdf5f-pqqqk\" (UID: \"ca89a42b-4d21-4d22-9181-e72d02a56659\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" Jun 07 09:11:32.710784 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.710725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" Jun 07 09:11:32.834159 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.833990 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk"] Jun 07 09:11:32.836670 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:32.836634 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca89a42b_4d21_4d22_9181_e72d02a56659.slice/crio-3847e850e6c7e200ce85ef8da6050ef7b36e37b0cb01f31066789544601cf63b WatchSource:0}: Error finding container 3847e850e6c7e200ce85ef8da6050ef7b36e37b0cb01f31066789544601cf63b: Status 404 returned error can't find the container with id 3847e850e6c7e200ce85ef8da6050ef7b36e37b0cb01f31066789544601cf63b Jun 07 09:11:32.902491 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:32.902468 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" event={"ID":"ca89a42b-4d21-4d22-9181-e72d02a56659","Type":"ContainerStarted","Data":"3847e850e6c7e200ce85ef8da6050ef7b36e37b0cb01f31066789544601cf63b"} Jun 07 09:11:33.624800 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:33.624772 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dfktq_d7f74ed1-5a63-4261-b482-e4a9e2b2e47b/dns-node-resolver/0.log" Jun 07 09:11:34.823872 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:34.823847 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-msxgz_795d8080-8304-4fe8-822b-723fc3feafbe/node-ca/0.log" Jun 07 09:11:34.909797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:34.909760 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" event={"ID":"ca89a42b-4d21-4d22-9181-e72d02a56659","Type":"ContainerStarted","Data":"59bb7587cd8cf97fec464feb4d27f6f1af836396f9cd81367971f124c37523c2"} Jun 07 09:11:34.909957 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:34.909816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" event={"ID":"ca89a42b-4d21-4d22-9181-e72d02a56659","Type":"ContainerStarted","Data":"b2fc1c92afb866221b12af793ae09f74719855d16accaaf2f80b1d12e688107b"} Jun 07 09:11:34.927652 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:34.927608 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-pqqqk" podStartSLOduration=1.5947524629999998 podStartE2EDuration="2.927593336s" totalImagesPullingTime="1.332840873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:34.925813755 +0000 UTC m=+207.070874675" watchObservedRunningTime="2026-06-07 09:11:34.927593336 +0000 UTC m=+207.072654256" Jun 07 09:11:34.935317 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:34.935295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:34.935413 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:34.935402 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:34.935456 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:34.935448 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls podName:a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:42.935436706 +0000 UTC m=+215.080497604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kkw42" (UID: "a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4") : secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:35.512972 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.512940 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-b67dh"] Jun 07 09:11:35.514898 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.514882 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.517487 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.517452 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 07 09:11:35.517591 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.517523 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 07 09:11:35.517591 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.517580 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2d842\"" Jun 07 09:11:35.518997 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.518978 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 07 09:11:35.519094 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.519022 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 07 09:11:35.522961 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.522938 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-b67dh"] Jun 07 09:11:35.640111 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.640082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d684c32b-c2fe-45f2-af3b-8f97e89dce54-signing-cabundle\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.640268 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.640162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d684c32b-c2fe-45f2-af3b-8f97e89dce54-signing-key\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.640268 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.640190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvwkf\" (UniqueName: \"kubernetes.io/projected/d684c32b-c2fe-45f2-af3b-8f97e89dce54-kube-api-access-hvwkf\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.741476 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.741437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d684c32b-c2fe-45f2-af3b-8f97e89dce54-signing-key\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.741476 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.741485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvwkf\" (UniqueName: \"kubernetes.io/projected/d684c32b-c2fe-45f2-af3b-8f97e89dce54-kube-api-access-hvwkf\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.741669 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.741511 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d684c32b-c2fe-45f2-af3b-8f97e89dce54-signing-cabundle\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.742174 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.742154 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d684c32b-c2fe-45f2-af3b-8f97e89dce54-signing-cabundle\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.743729 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.743711 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d684c32b-c2fe-45f2-af3b-8f97e89dce54-signing-key\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.749616 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.749593 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvwkf\" (UniqueName: \"kubernetes.io/projected/d684c32b-c2fe-45f2-af3b-8f97e89dce54-kube-api-access-hvwkf\") pod \"service-ca-7d74d578b8-b67dh\" (UID: \"d684c32b-c2fe-45f2-af3b-8f97e89dce54\") " pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.823618 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.823549 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" Jun 07 09:11:35.824737 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.824719 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gmnbx_dc36ff62-339f-4ea0-92fb-3f15cfebdd00/serve-healthcheck-canary/0.log" Jun 07 09:11:35.945027 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:35.944900 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-b67dh"] Jun 07 09:11:35.947373 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:35.947354 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd684c32b_c2fe_45f2_af3b_8f97e89dce54.slice/crio-73a20b37f02fa1ea811f728d503d5b2b19ccc040afcc3798bfe40f655216ddaa WatchSource:0}: Error finding container 73a20b37f02fa1ea811f728d503d5b2b19ccc040afcc3798bfe40f655216ddaa: Status 404 returned error can't find the container with id 73a20b37f02fa1ea811f728d503d5b2b19ccc040afcc3798bfe40f655216ddaa Jun 07 09:11:36.922600 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:36.922559 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" event={"ID":"d684c32b-c2fe-45f2-af3b-8f97e89dce54","Type":"ContainerStarted","Data":"0aaf3d94896fbd6efcd2ae3375bf9a600e19cc2d243ee017fb89a53caa508dbc"} Jun 07 09:11:36.923016 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:36.922654 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" event={"ID":"d684c32b-c2fe-45f2-af3b-8f97e89dce54","Type":"ContainerStarted","Data":"73a20b37f02fa1ea811f728d503d5b2b19ccc040afcc3798bfe40f655216ddaa"} Jun 07 09:11:36.940755 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:36.940705 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-b67dh" podStartSLOduration=1.940686747 podStartE2EDuration="1.940686747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:36.939759042 +0000 UTC m=+209.084819961" watchObservedRunningTime="2026-06-07 09:11:36.940686747 +0000 UTC m=+209.085747667" Jun 07 09:11:43.000070 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:43.000037 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:43.000434 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:43.000155 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:43.000434 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:11:43.000214 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls podName:a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:59.000200696 +0000 UTC m=+231.145261606 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-kkw42" (UID: "a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4") : secret "cluster-monitoring-operator-tls" not found Jun 07 09:11:54.193391 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.193360 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-njb2z"] Jun 07 09:11:54.196449 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.196430 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.199189 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.199164 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 07 09:11:54.200596 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.200577 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 07 09:11:54.200704 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.200673 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lk99w\"" Jun 07 09:11:54.207495 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.207472 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-njb2z"] Jun 07 09:11:54.266878 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.266854 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-lqxf6"] Jun 07 09:11:54.269717 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.269702 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-lqxf6" Jun 07 09:11:54.272872 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.272849 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 07 09:11:54.273005 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.272892 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9dxr4\"" Jun 07 09:11:54.273097 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.273081 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 07 09:11:54.279977 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.279956 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-lqxf6"] Jun 07 09:11:54.387076 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.387043 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8l2k\" (UniqueName: \"kubernetes.io/projected/ba660a44-ade6-4b34-99a6-229bb1872072-kube-api-access-c8l2k\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.387076 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.387075 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba660a44-ade6-4b34-99a6-229bb1872072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.387292 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.387094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba660a44-ade6-4b34-99a6-229bb1872072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.387292 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.387178 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfzhx\" (UniqueName: \"kubernetes.io/projected/657f9a89-a56d-4317-8197-9367dcb37cc8-kube-api-access-zfzhx\") pod \"downloads-6657875f97-lqxf6\" (UID: \"657f9a89-a56d-4317-8197-9367dcb37cc8\") " pod="openshift-console/downloads-6657875f97-lqxf6" Jun 07 09:11:54.387292 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.387221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba660a44-ade6-4b34-99a6-229bb1872072-data-volume\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.387292 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.387258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba660a44-ade6-4b34-99a6-229bb1872072-crio-socket\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.487999 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.487972 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba660a44-ade6-4b34-99a6-229bb1872072-data-volume\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.488179 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.488019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba660a44-ade6-4b34-99a6-229bb1872072-crio-socket\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.488179 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.488072 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8l2k\" (UniqueName: \"kubernetes.io/projected/ba660a44-ade6-4b34-99a6-229bb1872072-kube-api-access-c8l2k\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.488179 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.488088 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba660a44-ade6-4b34-99a6-229bb1872072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.488179 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.488103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba660a44-ade6-4b34-99a6-229bb1872072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.488179 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.488155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfzhx\" (UniqueName: \"kubernetes.io/projected/657f9a89-a56d-4317-8197-9367dcb37cc8-kube-api-access-zfzhx\") pod \"downloads-6657875f97-lqxf6\" (UID: \"657f9a89-a56d-4317-8197-9367dcb37cc8\") " pod="openshift-console/downloads-6657875f97-lqxf6" Jun 07 09:11:54.488421 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.488215 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba660a44-ade6-4b34-99a6-229bb1872072-crio-socket\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.488421 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.488365 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba660a44-ade6-4b34-99a6-229bb1872072-data-volume\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.488750 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.488733 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba660a44-ade6-4b34-99a6-229bb1872072-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.490546 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.490523 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba660a44-ade6-4b34-99a6-229bb1872072-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.497432 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.497409 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8l2k\" (UniqueName: \"kubernetes.io/projected/ba660a44-ade6-4b34-99a6-229bb1872072-kube-api-access-c8l2k\") pod \"insights-runtime-extractor-njb2z\" (UID: \"ba660a44-ade6-4b34-99a6-229bb1872072\") " pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.497526 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.497477 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfzhx\" (UniqueName: \"kubernetes.io/projected/657f9a89-a56d-4317-8197-9367dcb37cc8-kube-api-access-zfzhx\") pod \"downloads-6657875f97-lqxf6\" (UID: \"657f9a89-a56d-4317-8197-9367dcb37cc8\") " pod="openshift-console/downloads-6657875f97-lqxf6" Jun 07 09:11:54.505285 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.505265 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-njb2z" Jun 07 09:11:54.577763 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.577720 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-lqxf6" Jun 07 09:11:54.640626 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.640584 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-njb2z"] Jun 07 09:11:54.644563 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:54.644534 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba660a44_ade6_4b34_99a6_229bb1872072.slice/crio-9039a6475110f823754f8016c2b38c6e15f4a28a7d167e40b218a790a9747e49 WatchSource:0}: Error finding container 9039a6475110f823754f8016c2b38c6e15f4a28a7d167e40b218a790a9747e49: Status 404 returned error can't find the container with id 9039a6475110f823754f8016c2b38c6e15f4a28a7d167e40b218a790a9747e49 Jun 07 09:11:54.737043 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.736176 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-lqxf6"] Jun 07 09:11:54.738662 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:54.738627 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657f9a89_a56d_4317_8197_9367dcb37cc8.slice/crio-0df8213235626dd2a4c1c5e8528a864c69b9e7fab300171781da8fd5b4a0d008 WatchSource:0}: Error finding container 0df8213235626dd2a4c1c5e8528a864c69b9e7fab300171781da8fd5b4a0d008: Status 404 returned error can't find the container with id 0df8213235626dd2a4c1c5e8528a864c69b9e7fab300171781da8fd5b4a0d008 Jun 07 09:11:54.979748 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.979714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njb2z" event={"ID":"ba660a44-ade6-4b34-99a6-229bb1872072","Type":"ContainerStarted","Data":"592c9faa385dc3291a92f706d35bc2eb91c632eadab763f465c0a1a349a51cc3"} Jun 07 09:11:54.979902 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.979777 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njb2z" event={"ID":"ba660a44-ade6-4b34-99a6-229bb1872072","Type":"ContainerStarted","Data":"9039a6475110f823754f8016c2b38c6e15f4a28a7d167e40b218a790a9747e49"} Jun 07 09:11:54.980708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:54.980685 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-lqxf6" event={"ID":"657f9a89-a56d-4317-8197-9367dcb37cc8","Type":"ContainerStarted","Data":"0df8213235626dd2a4c1c5e8528a864c69b9e7fab300171781da8fd5b4a0d008"} Jun 07 09:11:55.987590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:55.987529 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njb2z" event={"ID":"ba660a44-ade6-4b34-99a6-229bb1872072","Type":"ContainerStarted","Data":"868dbcb472ac2c501fd851c46b010479e36b98631ebb0921c4fcb61b57fae83a"} Jun 07 09:11:57.997900 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:57.997863 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-njb2z" event={"ID":"ba660a44-ade6-4b34-99a6-229bb1872072","Type":"ContainerStarted","Data":"0f3fa9dbfc0b50d266d01f8be763b5fc7353fbcdd3b4b39c98b5e73b2724506b"} Jun 07 09:11:58.016422 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:58.016368 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-njb2z" podStartSLOduration=1.8066450600000001 podStartE2EDuration="4.016348571s" totalImagesPullingTime="2.209703511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:11:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:58.014029561 +0000 UTC m=+230.159090505" watchObservedRunningTime="2026-06-07 09:11:58.016348571 +0000 UTC m=+230.161409492" Jun 07 09:11:59.022458 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:59.022414 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:59.024658 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:59.024639 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-kkw42\" (UID: \"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:59.262164 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:59.262128 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" Jun 07 09:11:59.429100 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:11:59.429077 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42"] Jun 07 09:11:59.430910 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:11:59.430884 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00eb047_24f1_4eb0_8a2d_cf79c57e4cd4.slice/crio-55f426a850cfc70e15c3fca5934f9674b3207ea4d92fb74c7bf98ab787b9711a WatchSource:0}: Error finding container 55f426a850cfc70e15c3fca5934f9674b3207ea4d92fb74c7bf98ab787b9711a: Status 404 returned error can't find the container with id 55f426a850cfc70e15c3fca5934f9674b3207ea4d92fb74c7bf98ab787b9711a Jun 07 09:12:00.006100 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:00.006052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" event={"ID":"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4","Type":"ContainerStarted","Data":"55f426a850cfc70e15c3fca5934f9674b3207ea4d92fb74c7bf98ab787b9711a"} Jun 07 09:12:02.014101 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:02.014066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" event={"ID":"a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4","Type":"ContainerStarted","Data":"0be74731916b8ed4336184f442b241be074edda7d9e8d431157c12ceca93616b"} Jun 07 09:12:02.030579 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:02.030535 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-kkw42" podStartSLOduration=32.828148708 podStartE2EDuration="35.030520499s" totalImagesPullingTime="2.202371791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:02.030129083 +0000 UTC m=+234.175190002" watchObservedRunningTime="2026-06-07 09:12:02.030520499 +0000 UTC m=+234.175581417" Jun 07 09:12:04.231443 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.231411 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9"] Jun 07 09:12:04.261350 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.261312 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9"] Jun 07 09:12:04.261495 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.261419 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.263992 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.263967 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 07 09:12:04.264116 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.264088 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r2nxk\"" Jun 07 09:12:04.265368 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.265344 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 07 09:12:04.265525 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.265355 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 07 09:12:04.366937 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.366904 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv4rt\" (UniqueName: \"kubernetes.io/projected/73c0e393-a1fa-4493-b98c-4f729e7a2133-kube-api-access-xv4rt\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.367090 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.366958 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73c0e393-a1fa-4493-b98c-4f729e7a2133-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.367090 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.366988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.367090 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.367017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.468236 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.468201 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv4rt\" (UniqueName: \"kubernetes.io/projected/73c0e393-a1fa-4493-b98c-4f729e7a2133-kube-api-access-xv4rt\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.468236 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.468243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73c0e393-a1fa-4493-b98c-4f729e7a2133-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.468472 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.468270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.468472 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.468306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.468472 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:04.468403 2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 07 09:12:04.468472 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:04.468455 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-tls podName:73c0e393-a1fa-4493-b98c-4f729e7a2133 nodeName:}" failed. No retries permitted until 2026-06-07 09:12:04.968438513 +0000 UTC m=+237.113499427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-lnxf9" (UID: "73c0e393-a1fa-4493-b98c-4f729e7a2133") : secret "prometheus-operator-tls" not found Jun 07 09:12:04.469051 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.469017 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73c0e393-a1fa-4493-b98c-4f729e7a2133-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.471266 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.471231 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.478729 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.478704 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv4rt\" (UniqueName: \"kubernetes.io/projected/73c0e393-a1fa-4493-b98c-4f729e7a2133-kube-api-access-xv4rt\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.972344 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.972303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:04.974908 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:04.974880 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73c0e393-a1fa-4493-b98c-4f729e7a2133-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-lnxf9\" (UID: \"73c0e393-a1fa-4493-b98c-4f729e7a2133\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:05.172748 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.172712 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" Jun 07 09:12:05.324786 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.324760 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9"] Jun 07 09:12:05.327675 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:05.327643 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c0e393_a1fa_4493_b98c_4f729e7a2133.slice/crio-ee44d7a3c6c624b6ad259f7e511c1a22aa02a47ffb14125250ab8cc634d28bb0 WatchSource:0}: Error finding container ee44d7a3c6c624b6ad259f7e511c1a22aa02a47ffb14125250ab8cc634d28bb0: Status 404 returned error can't find the container with id ee44d7a3c6c624b6ad259f7e511c1a22aa02a47ffb14125250ab8cc634d28bb0 Jun 07 09:12:05.729957 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.729915 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b86b4c66c-znprx"] Jun 07 09:12:05.763812 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.763780 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b86b4c66c-znprx"] Jun 07 09:12:05.764025 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.763902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.769153 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.769128 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 07 09:12:05.769299 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.769283 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 07 09:12:05.769368 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.769335 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 07 09:12:05.770349 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.770284 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 07 09:12:05.770472 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.770372 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k2swh\"" Jun 07 09:12:05.770535 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.770502 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 07 09:12:05.774764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.774745 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 07 09:12:05.777998 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.777979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-config\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.778105 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.778008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-serving-cert\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.778105 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.778031 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-oauth-config\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.778105 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.778090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-service-ca\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.778296 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.778133 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-oauth-serving-cert\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.778296 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.778175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-trusted-ca-bundle\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.778296 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.778213 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjqgr\" (UniqueName: \"kubernetes.io/projected/49f4b2cb-9a9a-4b77-9868-2351008715b8-kube-api-access-cjqgr\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.879650 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.879599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-service-ca\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.879822 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.879683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-oauth-serving-cert\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.879822 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.879720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-trusted-ca-bundle\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.879822 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.879762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjqgr\" (UniqueName: \"kubernetes.io/projected/49f4b2cb-9a9a-4b77-9868-2351008715b8-kube-api-access-cjqgr\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.879822 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.879810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-config\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.880119 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.879836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-serving-cert\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.880119 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.879872 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-oauth-config\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.880678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.880447 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-service-ca\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.880795 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.880682 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-oauth-serving-cert\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.880862 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.880847 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-config\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.880918 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.880883 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-trusted-ca-bundle\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.883014 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.882990 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-oauth-config\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.883135 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.883116 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-serving-cert\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:05.887953 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:05.887917 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjqgr\" (UniqueName: \"kubernetes.io/projected/49f4b2cb-9a9a-4b77-9868-2351008715b8-kube-api-access-cjqgr\") pod \"console-5b86b4c66c-znprx\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:06.029440 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:06.029354 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" event={"ID":"73c0e393-a1fa-4493-b98c-4f729e7a2133","Type":"ContainerStarted","Data":"ee44d7a3c6c624b6ad259f7e511c1a22aa02a47ffb14125250ab8cc634d28bb0"} Jun 07 09:12:06.075658 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:06.075626 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:06.234277 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:06.234239 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b86b4c66c-znprx"] Jun 07 09:12:06.237004 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:06.236974 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f4b2cb_9a9a_4b77_9868_2351008715b8.slice/crio-a9df4002fa31f473246d2d81f1efba689963f0759eaa1924d9b37174a296ee4a WatchSource:0}: Error finding container a9df4002fa31f473246d2d81f1efba689963f0759eaa1924d9b37174a296ee4a: Status 404 returned error can't find the container with id a9df4002fa31f473246d2d81f1efba689963f0759eaa1924d9b37174a296ee4a Jun 07 09:12:07.038033 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:07.037976 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" event={"ID":"73c0e393-a1fa-4493-b98c-4f729e7a2133","Type":"ContainerStarted","Data":"8f46acf632f4a4eb81d51ed8846e5f4410e53374b3a071063867e93b382c1ddf"} Jun 07 09:12:07.038676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:07.038231 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" event={"ID":"73c0e393-a1fa-4493-b98c-4f729e7a2133","Type":"ContainerStarted","Data":"b946796180f00cb5d9551e7efea816db53e4442546a40be9d2dda216e8ee778c"} Jun 07 09:12:07.039761 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:07.039721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b86b4c66c-znprx" event={"ID":"49f4b2cb-9a9a-4b77-9868-2351008715b8","Type":"ContainerStarted","Data":"a9df4002fa31f473246d2d81f1efba689963f0759eaa1924d9b37174a296ee4a"} Jun 07 09:12:07.056613 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:07.056566 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-lnxf9" podStartSLOduration=1.724498337 podStartE2EDuration="3.056547194s" totalImagesPullingTime="1.332048857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:07.053919171 +0000 UTC m=+239.198980087" watchObservedRunningTime="2026-06-07 09:12:07.056547194 +0000 UTC m=+239.201608116" Jun 07 09:12:08.572124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.572076 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h"] Jun 07 09:12:08.575059 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.575037 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.578079 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.578057 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 07 09:12:08.578194 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.578175 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-thhhq\"" Jun 07 09:12:08.578361 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.578331 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 07 09:12:08.582238 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.582221 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-td5nj"] Jun 07 09:12:08.587314 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.586256 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h"] Jun 07 09:12:08.587314 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.586367 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.587314 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.586667 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895"] Jun 07 09:12:08.589557 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.589000 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 07 09:12:08.589557 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.589056 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 07 09:12:08.589557 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.589303 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 07 09:12:08.589557 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.589494 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pc6hm\"" Jun 07 09:12:08.590131 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.590115 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.593629 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.592950 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ngmtd\"" Jun 07 09:12:08.593629 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.593146 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 07 09:12:08.593629 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.593289 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 07 09:12:08.593629 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.593442 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.603988 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895"] Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604213 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-root\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b995a55a-31f5-40d9-b1e5-7bb42213e984-metrics-client-ca\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-sys\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604313 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-tls\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604395 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f1f148-57cb-411b-9590-66af20af1dd3-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-wtmp\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trcgr\" (UniqueName: \"kubernetes.io/projected/36f1f148-57cb-411b-9590-66af20af1dd3-kube-api-access-trcgr\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz4dd\" (UniqueName: \"kubernetes.io/projected/b995a55a-31f5-40d9-b1e5-7bb42213e984-kube-api-access-jz4dd\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-textfile\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.604747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.604629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-accelerators-collector-config\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-textfile\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-accelerators-collector-config\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705238 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-root\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b995a55a-31f5-40d9-b1e5-7bb42213e984-metrics-client-ca\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-sys\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705332 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-root\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjnn8\" (UniqueName: \"kubernetes.io/projected/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-api-access-zjnn8\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.705420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-tls\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705431 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/69fe17f9-556d-4f5c-9965-c4aa5a17357c-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705605 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69fe17f9-556d-4f5c-9965-c4aa5a17357c-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f1f148-57cb-411b-9590-66af20af1dd3-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705660 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-textfile\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705679 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-wtmp\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705771 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trcgr\" (UniqueName: \"kubernetes.io/projected/36f1f148-57cb-411b-9590-66af20af1dd3-kube-api-access-trcgr\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4dd\" (UniqueName: \"kubernetes.io/projected/b995a55a-31f5-40d9-b1e5-7bb42213e984-kube-api-access-jz4dd\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.705986 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b995a55a-31f5-40d9-b1e5-7bb42213e984-metrics-client-ca\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706698 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.706064 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-sys\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706698 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:08.706151 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 07 09:12:08.706698 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:08.706203 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-tls podName:b995a55a-31f5-40d9-b1e5-7bb42213e984 nodeName:}" failed. No retries permitted until 2026-06-07 09:12:09.20618561 +0000 UTC m=+241.351246510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-tls") pod "node-exporter-td5nj" (UID: "b995a55a-31f5-40d9-b1e5-7bb42213e984") : secret "node-exporter-tls" not found Jun 07 09:12:08.706698 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.706550 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-wtmp\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706698 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.706600 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-accelerators-collector-config\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.706698 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:08.706638 2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 07 09:12:08.706698 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:08.706677 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls podName:36f1f148-57cb-411b-9590-66af20af1dd3 nodeName:}" failed. No retries permitted until 2026-06-07 09:12:09.20666338 +0000 UTC m=+241.351724282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-lvr2h" (UID: "36f1f148-57cb-411b-9590-66af20af1dd3") : secret "openshift-state-metrics-tls" not found Jun 07 09:12:08.707221 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.707199 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36f1f148-57cb-411b-9590-66af20af1dd3-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.712259 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.711433 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.712259 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.712065 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.716294 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.716269 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz4dd\" (UniqueName: \"kubernetes.io/projected/b995a55a-31f5-40d9-b1e5-7bb42213e984-kube-api-access-jz4dd\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:08.717431 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.717378 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trcgr\" (UniqueName: \"kubernetes.io/projected/36f1f148-57cb-411b-9590-66af20af1dd3-kube-api-access-trcgr\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:08.806565 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.806532 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.806903 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.806878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjnn8\" (UniqueName: \"kubernetes.io/projected/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-api-access-zjnn8\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.807130 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.807115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.807259 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.807246 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/69fe17f9-556d-4f5c-9965-c4aa5a17357c-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.807395 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.807382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.807584 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.807569 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69fe17f9-556d-4f5c-9965-c4aa5a17357c-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.807959 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.807915 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.808225 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.808205 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/69fe17f9-556d-4f5c-9965-c4aa5a17357c-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.808312 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:08.808299 2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 07 09:12:08.808370 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:08.808360 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-tls podName:69fe17f9-556d-4f5c-9965-c4aa5a17357c nodeName:}" failed. No retries permitted until 2026-06-07 09:12:09.308343556 +0000 UTC m=+241.453404459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-tls") pod "kube-state-metrics-57bbf8bfb5-lg895" (UID: "69fe17f9-556d-4f5c-9965-c4aa5a17357c") : secret "kube-state-metrics-tls" not found Jun 07 09:12:08.808720 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.808703 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69fe17f9-556d-4f5c-9965-c4aa5a17357c-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.812522 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.812474 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:08.820640 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:08.820450 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjnn8\" (UniqueName: \"kubernetes.io/projected/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-api-access-zjnn8\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:09.212968 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:09.212903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-tls\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:09.213163 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:09.213041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:09.213227 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:09.213215 2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 07 09:12:09.213619 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:09.213284 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls podName:36f1f148-57cb-411b-9590-66af20af1dd3 nodeName:}" failed. No retries permitted until 2026-06-07 09:12:10.213265743 +0000 UTC m=+242.358326646 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-lvr2h" (UID: "36f1f148-57cb-411b-9590-66af20af1dd3") : secret "openshift-state-metrics-tls" not found Jun 07 09:12:09.215832 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:09.215804 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b995a55a-31f5-40d9-b1e5-7bb42213e984-node-exporter-tls\") pod \"node-exporter-td5nj\" (UID: \"b995a55a-31f5-40d9-b1e5-7bb42213e984\") " pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:09.313804 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:09.313754 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:09.316882 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:09.316851 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/69fe17f9-556d-4f5c-9965-c4aa5a17357c-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-lg895\" (UID: \"69fe17f9-556d-4f5c-9965-c4aa5a17357c\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:09.504298 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:09.504266 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-td5nj" Jun 07 09:12:09.514270 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:09.514241 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" Jun 07 09:12:10.223561 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.223520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:10.226040 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.226015 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/36f1f148-57cb-411b-9590-66af20af1dd3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-lvr2h\" (UID: \"36f1f148-57cb-411b-9590-66af20af1dd3\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:10.392772 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.392732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" Jun 07 09:12:10.568294 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.568259 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7694495584-gw8c4"] Jun 07 09:12:10.571182 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.571163 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.573976 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.573949 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 07 09:12:10.574088 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.573979 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 07 09:12:10.574327 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.574312 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1gamcqbje8gg5\"" Jun 07 09:12:10.574432 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.574413 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lvrw8\"" Jun 07 09:12:10.574481 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.574427 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 07 09:12:10.574535 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.574513 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 07 09:12:10.575303 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.575283 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 07 09:12:10.581532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.581510 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7694495584-gw8c4"] Jun 07 09:12:10.627690 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.627664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.627838 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.627708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.627838 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.627739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.627838 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.627790 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.627838 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.627815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beada5f8-a52b-4b4f-af31-7610d98fd074-metrics-client-ca\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.628076 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.627977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-grpc-tls\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.628076 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.628039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-tls\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.628076 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.628067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjrzk\" (UniqueName: \"kubernetes.io/projected/beada5f8-a52b-4b4f-af31-7610d98fd074-kube-api-access-gjrzk\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.729397 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.729358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.729397 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.729405 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.729641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.729451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.729641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.729487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.729641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.729514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beada5f8-a52b-4b4f-af31-7610d98fd074-metrics-client-ca\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.729641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.729572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-grpc-tls\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.729641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.729615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-tls\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.729641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.729638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjrzk\" (UniqueName: \"kubernetes.io/projected/beada5f8-a52b-4b4f-af31-7610d98fd074-kube-api-access-gjrzk\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.730422 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.730374 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beada5f8-a52b-4b4f-af31-7610d98fd074-metrics-client-ca\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.732735 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.732689 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.732838 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.732755 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.733108 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.733085 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-tls\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.733269 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.733227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.733375 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.733304 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.733547 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.733527 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/beada5f8-a52b-4b4f-af31-7610d98fd074-secret-grpc-tls\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.737914 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.737884 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjrzk\" (UniqueName: \"kubernetes.io/projected/beada5f8-a52b-4b4f-af31-7610d98fd074-kube-api-access-gjrzk\") pod \"thanos-querier-7694495584-gw8c4\" (UID: \"beada5f8-a52b-4b4f-af31-7610d98fd074\") " pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:10.883183 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:10.883089 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:14.788794 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.788761 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:12:14.792188 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.792148 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.795671 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.795641 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 07 09:12:14.795823 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.795681 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 07 09:12:14.795823 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.795726 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9n9cn2nsip9a1\"" Jun 07 09:12:14.795823 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.795682 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 07 09:12:14.795823 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.795641 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 07 09:12:14.796098 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.796034 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 07 09:12:14.797032 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797009 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 07 09:12:14.797131 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797100 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 07 09:12:14.797191 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797100 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xq7d8\"" Jun 07 09:12:14.797242 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797204 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 07 09:12:14.797372 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797355 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 07 09:12:14.797372 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797362 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 07 09:12:14.797490 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797422 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 07 09:12:14.797581 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797555 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 07 09:12:14.797581 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.797570 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 07 09:12:14.800162 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.800140 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 07 09:12:14.806003 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.805205 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 07 09:12:14.806470 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.806448 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:12:14.871601 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzkm6\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-kube-api-access-bzkm6\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.871768 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.871768 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871650 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.871768 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.871768 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.871768 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.871768 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871780 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871827 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.871960 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.872001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.872035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872078 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.872073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872415 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.872114 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872415 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.872155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872415 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.872212 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.872415 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.872254 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.973312 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzkm6\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-kube-api-access-bzkm6\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.973502 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.973502 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973383 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.973502 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973408 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.973773 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.973858 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.973858 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973821 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.973858 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974039 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974039 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974039 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974039 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.973977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974039 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974273 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974045 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974273 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974273 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974117 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974273 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974273 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974273 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.974563 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974284 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.975024 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.974697 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.979611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.978282 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.979611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.978354 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.979611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.978946 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.979611 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.979139 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.979990 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.979969 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.980173 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.980151 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.980342 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.980323 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.980961 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.980904 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.981649 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.981625 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.982835 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.983329 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.983343 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.983536 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.984017 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.984068 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.984142 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.984321 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.984830 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.984618 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzkm6\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-kube-api-access-bzkm6\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:14.985494 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:14.985477 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:15.108150 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:15.108050 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:16.417558 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:16.417527 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb995a55a_31f5_40d9_b1e5_7bb42213e984.slice/crio-a9b19bbae11c50d254cb92920fed8a65980f7308c4c6413372d65e8f1e37e521 WatchSource:0}: Error finding container a9b19bbae11c50d254cb92920fed8a65980f7308c4c6413372d65e8f1e37e521: Status 404 returned error can't find the container with id a9b19bbae11c50d254cb92920fed8a65980f7308c4c6413372d65e8f1e37e521 Jun 07 09:12:16.590727 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:16.590644 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h"] Jun 07 09:12:16.591347 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:16.591315 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f1f148_57cb_411b_9590_66af20af1dd3.slice/crio-84bed2ae851b3c0246f50920b9888a0e7da4d33e853e501d1135e0224dc4bbc6 WatchSource:0}: Error finding container 84bed2ae851b3c0246f50920b9888a0e7da4d33e853e501d1135e0224dc4bbc6: Status 404 returned error can't find the container with id 84bed2ae851b3c0246f50920b9888a0e7da4d33e853e501d1135e0224dc4bbc6 Jun 07 09:12:16.821088 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:16.820812 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895"] Jun 07 09:12:16.823749 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:16.823124 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69fe17f9_556d_4f5c_9965_c4aa5a17357c.slice/crio-d868fdfe01368d1bdb17f40516dffddeef684e66f10e6d7e41341b3bd5c3f813 WatchSource:0}: Error finding container d868fdfe01368d1bdb17f40516dffddeef684e66f10e6d7e41341b3bd5c3f813: Status 404 returned error can't find the container with id d868fdfe01368d1bdb17f40516dffddeef684e66f10e6d7e41341b3bd5c3f813 Jun 07 09:12:16.827779 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:16.826422 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7694495584-gw8c4"] Jun 07 09:12:16.830117 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:16.830090 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:12:16.836368 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:16.836338 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b31932c_4e6c_49c0_847f_4fbc6dd4a477.slice/crio-7477ca09cfbf45d63543e596648122b168b9fdb4db6d1260530a6db7323cf323 WatchSource:0}: Error finding container 7477ca09cfbf45d63543e596648122b168b9fdb4db6d1260530a6db7323cf323: Status 404 returned error can't find the container with id 7477ca09cfbf45d63543e596648122b168b9fdb4db6d1260530a6db7323cf323 Jun 07 09:12:17.085846 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.085784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerStarted","Data":"7477ca09cfbf45d63543e596648122b168b9fdb4db6d1260530a6db7323cf323"} Jun 07 09:12:17.088475 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.088446 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" event={"ID":"36f1f148-57cb-411b-9590-66af20af1dd3","Type":"ContainerStarted","Data":"8ea9daa905459de19a8dcfaa2e42371b4cf285a00f6a83edf7871ae8c1829ea0"} Jun 07 09:12:17.088597 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.088506 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" event={"ID":"36f1f148-57cb-411b-9590-66af20af1dd3","Type":"ContainerStarted","Data":"3adfe812c54591d7a138a47eae71ae0ddbdc8af9501638c2303eb34f3bd94223"} Jun 07 09:12:17.088597 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.088533 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" event={"ID":"36f1f148-57cb-411b-9590-66af20af1dd3","Type":"ContainerStarted","Data":"84bed2ae851b3c0246f50920b9888a0e7da4d33e853e501d1135e0224dc4bbc6"} Jun 07 09:12:17.090663 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.090611 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" event={"ID":"beada5f8-a52b-4b4f-af31-7610d98fd074","Type":"ContainerStarted","Data":"8dd6d2a28cd38c6c07ced6de0de4914998b322fcfca7c9d98a31305183d0c36e"} Jun 07 09:12:17.093285 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.092692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b86b4c66c-znprx" event={"ID":"49f4b2cb-9a9a-4b77-9868-2351008715b8","Type":"ContainerStarted","Data":"71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66"} Jun 07 09:12:17.094860 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.094808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-td5nj" event={"ID":"b995a55a-31f5-40d9-b1e5-7bb42213e984","Type":"ContainerStarted","Data":"a9b19bbae11c50d254cb92920fed8a65980f7308c4c6413372d65e8f1e37e521"} Jun 07 09:12:17.098059 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.096567 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-lqxf6" event={"ID":"657f9a89-a56d-4317-8197-9367dcb37cc8","Type":"ContainerStarted","Data":"3813d4d8f7218d1aade003ac8fd2bb9426a0b2a0ef13a2c9b37343e87f9462ce"} Jun 07 09:12:17.098059 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.097154 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-lqxf6" Jun 07 09:12:17.098965 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.098540 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" event={"ID":"69fe17f9-556d-4f5c-9965-c4aa5a17357c","Type":"ContainerStarted","Data":"d868fdfe01368d1bdb17f40516dffddeef684e66f10e6d7e41341b3bd5c3f813"} Jun 07 09:12:17.109747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.109723 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-lqxf6" Jun 07 09:12:17.112043 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.111901 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b86b4c66c-znprx" podStartSLOduration=1.934644618 podStartE2EDuration="12.111883446s" totalImagesPullingTime="10.177238828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:17.110771368 +0000 UTC m=+249.255832290" watchObservedRunningTime="2026-06-07 09:12:17.111883446 +0000 UTC m=+249.256944366" Jun 07 09:12:17.128938 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.128873 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-lqxf6" podStartSLOduration=1.417337488 podStartE2EDuration="23.128856658s" totalImagesPullingTime="21.71151917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:17.12781026 +0000 UTC m=+249.272871193" watchObservedRunningTime="2026-06-07 09:12:17.128856658 +0000 UTC m=+249.273917579" Jun 07 09:12:17.202937 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:17.202891 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b86b4c66c-znprx"] Jun 07 09:12:18.115115 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:18.114548 2578 generic.go:356] "Generic (PLEG): container finished" podID="b995a55a-31f5-40d9-b1e5-7bb42213e984" containerID="086ecb58e930bb1799d534408ff0b5f23c51b3039f88324fd4f5e199616d28a0" exitCode=0 Jun 07 09:12:18.115115 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:18.114653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-td5nj" event={"ID":"b995a55a-31f5-40d9-b1e5-7bb42213e984","Type":"ContainerDied","Data":"086ecb58e930bb1799d534408ff0b5f23c51b3039f88324fd4f5e199616d28a0"} Jun 07 09:12:19.121645 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:19.121609 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-td5nj" event={"ID":"b995a55a-31f5-40d9-b1e5-7bb42213e984","Type":"ContainerStarted","Data":"7c20c6b50c31d229ca7ab727acdfca35b500a4fa60ea7ad4be2bcfcd89302ff7"} Jun 07 09:12:20.235178 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:20.235056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:12:20.238030 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:20.237978 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfbdde5e-d359-422e-bf75-55f3221bd8f3-metrics-certs\") pod \"network-metrics-daemon-p4shp\" (UID: \"dfbdde5e-d359-422e-bf75-55f3221bd8f3\") " pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:12:20.498056 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:20.497948 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jh2fl\"" Jun 07 09:12:20.506320 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:20.506262 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-p4shp" Jun 07 09:12:21.224625 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:21.224060 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-p4shp"] Jun 07 09:12:21.229347 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:21.229228 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfbdde5e_d359_422e_bf75_55f3221bd8f3.slice/crio-597dd7ab744771aed8235a2bc3105872a3b77dee60fd2af32fffaeb6da230b91 WatchSource:0}: Error finding container 597dd7ab744771aed8235a2bc3105872a3b77dee60fd2af32fffaeb6da230b91: Status 404 returned error can't find the container with id 597dd7ab744771aed8235a2bc3105872a3b77dee60fd2af32fffaeb6da230b91 Jun 07 09:12:22.158048 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.157850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-td5nj" event={"ID":"b995a55a-31f5-40d9-b1e5-7bb42213e984","Type":"ContainerStarted","Data":"7013520e368f78c22b5c883a875d2c3e1ae898107ac15b62ec2bab0c48a3b3b4"} Jun 07 09:12:22.159676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.159643 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p4shp" event={"ID":"dfbdde5e-d359-422e-bf75-55f3221bd8f3","Type":"ContainerStarted","Data":"597dd7ab744771aed8235a2bc3105872a3b77dee60fd2af32fffaeb6da230b91"} Jun 07 09:12:22.162379 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.162343 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" event={"ID":"69fe17f9-556d-4f5c-9965-c4aa5a17357c","Type":"ContainerStarted","Data":"c0db013558dbf5b895a2b7dbe6559f773c0fd400e6be2331d898490b80f7d03b"} Jun 07 09:12:22.162504 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.162403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" event={"ID":"69fe17f9-556d-4f5c-9965-c4aa5a17357c","Type":"ContainerStarted","Data":"676ed09e9a381e79f415df5c2f30b15729f9a4c57c1cb8bb6554590812638955"} Jun 07 09:12:22.162504 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.162436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" event={"ID":"69fe17f9-556d-4f5c-9965-c4aa5a17357c","Type":"ContainerStarted","Data":"e29fb06f77972b76a9564b4e1107904157cd8f28dd83e0dca82f25074a677c2d"} Jun 07 09:12:22.164520 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.164493 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" exitCode=0 Jun 07 09:12:22.164696 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.164575 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerDied","Data":"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26"} Jun 07 09:12:22.167595 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.167544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" event={"ID":"36f1f148-57cb-411b-9590-66af20af1dd3","Type":"ContainerStarted","Data":"23ba4f88b6bf991496e3217f369af077d3e12ce6585714b55ec107bf8bda67de"} Jun 07 09:12:22.170820 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.170745 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" event={"ID":"beada5f8-a52b-4b4f-af31-7610d98fd074","Type":"ContainerStarted","Data":"a4e1c255f47571640808a9b7f3a0a3d8219f5257972cb87c055c73adde2e88c5"} Jun 07 09:12:22.170820 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.170811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" event={"ID":"beada5f8-a52b-4b4f-af31-7610d98fd074","Type":"ContainerStarted","Data":"0f441f8555b894b7da100653c3c40f2afed302ded8ca3a7403aebf904c50846b"} Jun 07 09:12:22.170995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.170851 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" event={"ID":"beada5f8-a52b-4b4f-af31-7610d98fd074","Type":"ContainerStarted","Data":"7d1b2f46cc32b13656b564073dce9f82e105eda0d70022b427e55c5ff45330b2"} Jun 07 09:12:22.179808 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.179767 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-td5nj" podStartSLOduration=13.16812564 podStartE2EDuration="14.179754471s" totalImagesPullingTime="920.582741ms" totalInitContainerRuntime="91.04609ms" isStatefulPod=true podCreationTimestamp="2026-06-07 09:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:22.177674642 +0000 UTC m=+254.322735596" watchObservedRunningTime="2026-06-07 09:12:22.179754471 +0000 UTC m=+254.324815391" Jun 07 09:12:22.221152 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.221103 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-lg895" podStartSLOduration=10.002511155 podStartE2EDuration="14.221084749s" totalImagesPullingTime="4.218573594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:22.219656407 +0000 UTC m=+254.364717328" watchObservedRunningTime="2026-06-07 09:12:22.221084749 +0000 UTC m=+254.366145668" Jun 07 09:12:22.239973 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:22.239907 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-lvr2h" podStartSLOduration=9.918005796 podStartE2EDuration="14.239887716s" totalImagesPullingTime="4.32188192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:22.238420611 +0000 UTC m=+254.383481552" watchObservedRunningTime="2026-06-07 09:12:22.239887716 +0000 UTC m=+254.384948636" Jun 07 09:12:23.178884 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:23.178846 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p4shp" event={"ID":"dfbdde5e-d359-422e-bf75-55f3221bd8f3","Type":"ContainerStarted","Data":"797e02184cfeafa4c5664b8cc976a536ce4c4495be59e2bd076dcb9609103840"} Jun 07 09:12:23.187290 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:23.184438 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" event={"ID":"beada5f8-a52b-4b4f-af31-7610d98fd074","Type":"ContainerStarted","Data":"743d65702f6bf1d29b9d847c7771d8386d5741b91188ae21dadf97443ef25ced"} Jun 07 09:12:24.194684 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:24.194554 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" event={"ID":"beada5f8-a52b-4b4f-af31-7610d98fd074","Type":"ContainerStarted","Data":"a31d2bd9e8753d7d681cfaec7c34aacd12543e1d1f744724277ae28d7b8b2b86"} Jun 07 09:12:24.194684 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:24.194643 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" event={"ID":"beada5f8-a52b-4b4f-af31-7610d98fd074","Type":"ContainerStarted","Data":"98dcb78d768356354ab2f7ce5985f581853127dcefb89a22b8b89481c17d380c"} Jun 07 09:12:24.195759 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:24.195723 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:24.198257 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:24.198222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-p4shp" event={"ID":"dfbdde5e-d359-422e-bf75-55f3221bd8f3","Type":"ContainerStarted","Data":"495f399121f68230a59966f17393f3b959927312c6f1d9f42c12c616c43119e1"} Jun 07 09:12:24.219694 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:24.219641 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" podStartSLOduration=8.716221325 podStartE2EDuration="14.219621188s" totalImagesPullingTime="5.503399863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:24.216565751 +0000 UTC m=+256.361626673" watchObservedRunningTime="2026-06-07 09:12:24.219621188 +0000 UTC m=+256.364682108" Jun 07 09:12:24.232357 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:24.232307 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-p4shp" podStartSLOduration=254.640519832 podStartE2EDuration="4m16.232287534s" totalImagesPullingTime="1.591767702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:24.231321762 +0000 UTC m=+256.376382685" watchObservedRunningTime="2026-06-07 09:12:24.232287534 +0000 UTC m=+256.377348453" Jun 07 09:12:26.076941 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:26.076892 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:26.212477 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:26.212451 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7694495584-gw8c4" Jun 07 09:12:27.214367 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:27.214284 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerStarted","Data":"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b"} Jun 07 09:12:27.214833 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:27.214399 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerStarted","Data":"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a"} Jun 07 09:12:27.214833 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:27.214449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerStarted","Data":"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc"} Jun 07 09:12:27.214833 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:27.214490 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerStarted","Data":"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd"} Jun 07 09:12:27.214833 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:27.214529 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerStarted","Data":"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3"} Jun 07 09:12:27.214833 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:27.214575 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerStarted","Data":"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa"} Jun 07 09:12:27.240151 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:27.240111 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.911254131 podStartE2EDuration="13.24009311s" totalImagesPullingTime="8.300156901s" totalInitContainerRuntime="28.682078ms" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:27.239426703 +0000 UTC m=+259.384487646" watchObservedRunningTime="2026-06-07 09:12:27.24009311 +0000 UTC m=+259.385154028" Jun 07 09:12:30.108490 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:30.108457 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:12:42.275318 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:42.275283 2578 generic.go:356] "Generic (PLEG): container finished" podID="da36a86e-4fab-4dad-85e3-0c9232d4a5f0" containerID="9bcba4d47b38869ba8da7edda659a893c4554880c18f60d45b72c5b8a5cd8d4b" exitCode=0 Jun 07 09:12:42.275701 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:42.275350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" event={"ID":"da36a86e-4fab-4dad-85e3-0c9232d4a5f0","Type":"ContainerDied","Data":"9bcba4d47b38869ba8da7edda659a893c4554880c18f60d45b72c5b8a5cd8d4b"} Jun 07 09:12:42.275746 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:42.275721 2578 scope.go:117] "RemoveContainer" containerID="9bcba4d47b38869ba8da7edda659a893c4554880c18f60d45b72c5b8a5cd8d4b" Jun 07 09:12:43.282334 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:43.282300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-qw7sk" event={"ID":"da36a86e-4fab-4dad-85e3-0c9232d4a5f0","Type":"ContainerStarted","Data":"5a8771ce0979367b9c201b344a407d2e34bfdd78e8120863900e21db3a2eef3d"} Jun 07 09:12:44.144691 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:44.144654 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b86b4c66c-znprx" podUID="49f4b2cb-9a9a-4b77-9868-2351008715b8" containerName="console" containerID="cri-o://71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66" gracePeriod=15 Jun 07 09:12:48.774263 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:48.774224 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" podUID="3c21300a-c19c-4af6-aaa0-f4927661a250" Jun 07 09:12:49.311251 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:49.311222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:12:51.320746 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:51.320718 2578 generic.go:356] "Generic (PLEG): container finished" podID="61c5adf4-9ecc-4355-a255-2aa644c130b9" containerID="5ac8ebd62aae91b18a30e56c7fd4c89a0eeaf1800e9f0740bf7201dca320268c" exitCode=0 Jun 07 09:12:51.321082 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:51.320786 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-vf2x8" event={"ID":"61c5adf4-9ecc-4355-a255-2aa644c130b9","Type":"ContainerDied","Data":"5ac8ebd62aae91b18a30e56c7fd4c89a0eeaf1800e9f0740bf7201dca320268c"} Jun 07 09:12:51.321082 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:51.321048 2578 scope.go:117] "RemoveContainer" containerID="5ac8ebd62aae91b18a30e56c7fd4c89a0eeaf1800e9f0740bf7201dca320268c" Jun 07 09:12:52.327170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.327131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-vf2x8" event={"ID":"61c5adf4-9ecc-4355-a255-2aa644c130b9","Type":"ContainerStarted","Data":"e537b2cd8ca586ed63d94bf0c606198d984fe15ca1d974c4135d85ee350ce2a4"} Jun 07 09:12:52.328708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.328656 2578 generic.go:356] "Generic (PLEG): container finished" podID="55194eb9-d80a-43f9-aa3e-a1355156a565" containerID="fb7b9834d9a812508f68d13a01da38e12b53155a3094031bf4d9babf2ef67fbc" exitCode=0 Jun 07 09:12:52.328820 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.328726 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" event={"ID":"55194eb9-d80a-43f9-aa3e-a1355156a565","Type":"ContainerDied","Data":"fb7b9834d9a812508f68d13a01da38e12b53155a3094031bf4d9babf2ef67fbc"} Jun 07 09:12:52.329023 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.329009 2578 scope.go:117] "RemoveContainer" containerID="fb7b9834d9a812508f68d13a01da38e12b53155a3094031bf4d9babf2ef67fbc" Jun 07 09:12:52.629377 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.629278 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:12:52.631775 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.631753 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"image-registry-6c97d5c65c-w8rrw\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:12:52.730099 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.730069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:12:52.732348 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.732316 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa52f142-0b4a-40a4-a7e3-3445e168731a-metrics-tls\") pod \"dns-default-gqbxw\" (UID: \"fa52f142-0b4a-40a4-a7e3-3445e168731a\") " pod="openshift-dns/dns-default-gqbxw" Jun 07 09:12:52.796414 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.796390 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmvt5\"" Jun 07 09:12:52.803910 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.803889 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gqbxw" Jun 07 09:12:52.915276 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.915255 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sqrsh\"" Jun 07 09:12:52.922636 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.922616 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:12:52.948270 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:52.948235 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gqbxw"] Jun 07 09:12:52.948556 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:52.948536 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa52f142_0b4a_40a4_a7e3_3445e168731a.slice/crio-da6c1ba85e546c26fc7352500dc484fbfd61c1d518e9e0e260c384e01daec6aa WatchSource:0}: Error finding container da6c1ba85e546c26fc7352500dc484fbfd61c1d518e9e0e260c384e01daec6aa: Status 404 returned error can't find the container with id da6c1ba85e546c26fc7352500dc484fbfd61c1d518e9e0e260c384e01daec6aa Jun 07 09:12:53.059486 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:53.059460 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c97d5c65c-w8rrw"] Jun 07 09:12:53.060619 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:12:53.060599 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c21300a_c19c_4af6_aaa0_f4927661a250.slice/crio-a5437b4c4082c66395880dac2f60eced58602b9740787220ba194b89907f8d83 WatchSource:0}: Error finding container a5437b4c4082c66395880dac2f60eced58602b9740787220ba194b89907f8d83: Status 404 returned error can't find the container with id a5437b4c4082c66395880dac2f60eced58602b9740787220ba194b89907f8d83 Jun 07 09:12:53.334435 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:53.334403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gqbxw" event={"ID":"fa52f142-0b4a-40a4-a7e3-3445e168731a","Type":"ContainerStarted","Data":"da6c1ba85e546c26fc7352500dc484fbfd61c1d518e9e0e260c384e01daec6aa"} Jun 07 09:12:53.335996 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:53.335970 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" event={"ID":"3c21300a-c19c-4af6-aaa0-f4927661a250","Type":"ContainerStarted","Data":"7b914a025f60111cccfdeeee5315a6bae1fbd381e78c2ec2b9e4c0b271a7ac33"} Jun 07 09:12:53.336126 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:53.336014 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" event={"ID":"3c21300a-c19c-4af6-aaa0-f4927661a250","Type":"ContainerStarted","Data":"a5437b4c4082c66395880dac2f60eced58602b9740787220ba194b89907f8d83"} Jun 07 09:12:53.336126 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:53.336106 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:12:53.337838 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:53.337814 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-wrccv" event={"ID":"55194eb9-d80a-43f9-aa3e-a1355156a565","Type":"ContainerStarted","Data":"2d90962b111dea2a01567ec08fa58ef44275821c08a5774494ba93642fc5e1e1"} Jun 07 09:12:53.359178 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:53.359137 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" podStartSLOduration=285.359122359 podStartE2EDuration="4m45.359122359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:53.35726242 +0000 UTC m=+285.502323351" watchObservedRunningTime="2026-06-07 09:12:53.359122359 +0000 UTC m=+285.504183278" Jun 07 09:12:56.350865 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:56.350831 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gqbxw" event={"ID":"fa52f142-0b4a-40a4-a7e3-3445e168731a","Type":"ContainerStarted","Data":"04e77fdcdf036c7707aed725a543af1404118d833fa19a8c50a15fad39f90b40"} Jun 07 09:12:56.351312 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:56.350877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gqbxw" event={"ID":"fa52f142-0b4a-40a4-a7e3-3445e168731a","Type":"ContainerStarted","Data":"e0e89ae23070705c44a2a375b8a83739c463dc07daab8468aedcb0cd427e6661"} Jun 07 09:12:56.351312 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:56.350974 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gqbxw" Jun 07 09:12:56.368873 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:56.368829 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gqbxw" podStartSLOduration=252.077338811 podStartE2EDuration="4m14.368815238s" totalImagesPullingTime="2.291476427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:56.366855304 +0000 UTC m=+288.511916224" watchObservedRunningTime="2026-06-07 09:12:56.368815238 +0000 UTC m=+288.513876158" Jun 07 09:12:59.341811 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.341792 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b86b4c66c-znprx_49f4b2cb-9a9a-4b77-9868-2351008715b8/console/0.log" Jun 07 09:12:59.361797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.361773 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:59.364045 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.364030 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b86b4c66c-znprx_49f4b2cb-9a9a-4b77-9868-2351008715b8/console/0.log" Jun 07 09:12:59.364166 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.364066 2578 generic.go:356] "Generic (PLEG): container finished" podID="49f4b2cb-9a9a-4b77-9868-2351008715b8" containerID="71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66" exitCode=137 Jun 07 09:12:59.364166 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.364104 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b86b4c66c-znprx" Jun 07 09:12:59.364279 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.364159 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b86b4c66c-znprx" event={"ID":"49f4b2cb-9a9a-4b77-9868-2351008715b8","Type":"ContainerDied","Data":"71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66"} Jun 07 09:12:59.364279 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.364211 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b86b4c66c-znprx" event={"ID":"49f4b2cb-9a9a-4b77-9868-2351008715b8","Type":"ContainerDied","Data":"a9df4002fa31f473246d2d81f1efba689963f0759eaa1924d9b37174a296ee4a"} Jun 07 09:12:59.364279 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.364225 2578 scope.go:117] "RemoveContainer" containerID="71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66" Jun 07 09:12:59.377556 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.377540 2578 scope.go:117] "RemoveContainer" containerID="71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66" Jun 07 09:12:59.377837 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:12:59.377810 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66\": container with ID starting with 71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66 not found: ID does not exist" containerID="71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66" Jun 07 09:12:59.377883 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.377847 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66"} err="failed to get container status \"71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66\": rpc error: code = NotFound desc = could not find container \"71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66\": container with ID starting with 71502985a876765baa1bc783393b86a057dd9319e6e784ad19e4a913e4c70a66 not found: ID does not exist" Jun 07 09:12:59.391032 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391010 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjqgr\" (UniqueName: \"kubernetes.io/projected/49f4b2cb-9a9a-4b77-9868-2351008715b8-kube-api-access-cjqgr\") pod \"49f4b2cb-9a9a-4b77-9868-2351008715b8\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " Jun 07 09:12:59.391120 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391074 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-serving-cert\") pod \"49f4b2cb-9a9a-4b77-9868-2351008715b8\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " Jun 07 09:12:59.391120 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391104 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-config\") pod \"49f4b2cb-9a9a-4b77-9868-2351008715b8\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " Jun 07 09:12:59.391196 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391143 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-service-ca\") pod \"49f4b2cb-9a9a-4b77-9868-2351008715b8\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " Jun 07 09:12:59.391196 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391168 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-oauth-serving-cert\") pod \"49f4b2cb-9a9a-4b77-9868-2351008715b8\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " Jun 07 09:12:59.391355 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391208 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-oauth-config\") pod \"49f4b2cb-9a9a-4b77-9868-2351008715b8\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " Jun 07 09:12:59.391355 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391240 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-trusted-ca-bundle\") pod \"49f4b2cb-9a9a-4b77-9868-2351008715b8\" (UID: \"49f4b2cb-9a9a-4b77-9868-2351008715b8\") " Jun 07 09:12:59.391533 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391508 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-config" (OuterVolumeSpecName: "console-config") pod "49f4b2cb-9a9a-4b77-9868-2351008715b8" (UID: "49f4b2cb-9a9a-4b77-9868-2351008715b8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:12:59.391624 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391542 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-service-ca" (OuterVolumeSpecName: "service-ca") pod "49f4b2cb-9a9a-4b77-9868-2351008715b8" (UID: "49f4b2cb-9a9a-4b77-9868-2351008715b8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:12:59.391855 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391833 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "49f4b2cb-9a9a-4b77-9868-2351008715b8" (UID: "49f4b2cb-9a9a-4b77-9868-2351008715b8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:12:59.391905 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.391851 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "49f4b2cb-9a9a-4b77-9868-2351008715b8" (UID: "49f4b2cb-9a9a-4b77-9868-2351008715b8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:12:59.393360 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.393337 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "49f4b2cb-9a9a-4b77-9868-2351008715b8" (UID: "49f4b2cb-9a9a-4b77-9868-2351008715b8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:12:59.393464 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.393407 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49f4b2cb-9a9a-4b77-9868-2351008715b8-kube-api-access-cjqgr" (OuterVolumeSpecName: "kube-api-access-cjqgr") pod "49f4b2cb-9a9a-4b77-9868-2351008715b8" (UID: "49f4b2cb-9a9a-4b77-9868-2351008715b8"). InnerVolumeSpecName "kube-api-access-cjqgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:12:59.393515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.393465 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "49f4b2cb-9a9a-4b77-9868-2351008715b8" (UID: "49f4b2cb-9a9a-4b77-9868-2351008715b8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:12:59.492135 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.492105 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjqgr\" (UniqueName: \"kubernetes.io/projected/49f4b2cb-9a9a-4b77-9868-2351008715b8-kube-api-access-cjqgr\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:12:59.492135 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.492134 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-serving-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:12:59.492135 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.492143 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-config\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:12:59.492345 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.492152 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-service-ca\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:12:59.492345 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.492160 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-oauth-serving-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:12:59.492345 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.492171 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49f4b2cb-9a9a-4b77-9868-2351008715b8-console-oauth-config\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:12:59.492345 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.492182 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49f4b2cb-9a9a-4b77-9868-2351008715b8-trusted-ca-bundle\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:12:59.687178 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.687150 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b86b4c66c-znprx"] Jun 07 09:12:59.690961 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:12:59.690919 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b86b4c66c-znprx"] Jun 07 09:13:00.298661 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:00.298631 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49f4b2cb-9a9a-4b77-9868-2351008715b8" path="/var/lib/kubelet/pods/49f4b2cb-9a9a-4b77-9868-2351008715b8/volumes" Jun 07 09:13:06.356777 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:06.356750 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gqbxw" Jun 07 09:13:08.255115 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:08.255084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:13:08.256634 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:08.256615 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:13:08.258424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:08.258403 2578 kubelet.go:1648] "Image garbage collection succeeded" Jun 07 09:13:12.927385 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:12.927348 2578 patch_prober.go:28] interesting pod/image-registry-6c97d5c65c-w8rrw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:13:12.928603 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:12.927405 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" podUID="3c21300a-c19c-4af6-aaa0-f4927661a250" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:13:14.345842 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:14.345813 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:13:15.109044 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:15.108998 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:15.128760 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:15.128729 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:15.445851 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:15.445779 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:22.338964 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:22.338919 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c97d5c65c-w8rrw"] Jun 07 09:13:33.151659 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.151614 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:13:33.152293 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.152011 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="prometheus" containerID="cri-o://7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" gracePeriod=600 Jun 07 09:13:33.152401 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.152365 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy" containerID="cri-o://c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" gracePeriod=600 Jun 07 09:13:33.152544 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.152419 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy-web" containerID="cri-o://5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" gracePeriod=600 Jun 07 09:13:33.152544 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.152530 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="thanos-sidecar" containerID="cri-o://95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" gracePeriod=600 Jun 07 09:13:33.152650 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.152555 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy-thanos" containerID="cri-o://cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" gracePeriod=600 Jun 07 09:13:33.152650 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.152614 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="config-reloader" containerID="cri-o://e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" gracePeriod=600 Jun 07 09:13:33.419966 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.419915 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.511393 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511362 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" exitCode=0 Jun 07 09:13:33.511393 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511388 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" exitCode=0 Jun 07 09:13:33.511393 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511397 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" exitCode=0 Jun 07 09:13:33.511393 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511404 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" exitCode=0 Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511409 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" exitCode=0 Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511414 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" exitCode=0 Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerDied","Data":"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b"} Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511449 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511501 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerDied","Data":"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a"} Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511529 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerDied","Data":"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc"} Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511554 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerDied","Data":"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd"} Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511572 2578 scope.go:117] "RemoveContainer" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511579 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerDied","Data":"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3"} Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511680 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerDied","Data":"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa"} Jun 07 09:13:33.511708 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.511704 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b31932c-4e6c-49c0-847f-4fbc6dd4a477","Type":"ContainerDied","Data":"7477ca09cfbf45d63543e596648122b168b9fdb4db6d1260530a6db7323cf323"} Jun 07 09:13:33.520359 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.520330 2578 scope.go:117] "RemoveContainer" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" Jun 07 09:13:33.527967 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.527938 2578 scope.go:117] "RemoveContainer" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" Jun 07 09:13:33.537075 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.536088 2578 scope.go:117] "RemoveContainer" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" Jun 07 09:13:33.544183 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.544163 2578 scope.go:117] "RemoveContainer" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" Jun 07 09:13:33.551274 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.551254 2578 scope.go:117] "RemoveContainer" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" Jun 07 09:13:33.558891 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.558873 2578 scope.go:117] "RemoveContainer" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" Jun 07 09:13:33.565905 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.565889 2578 scope.go:117] "RemoveContainer" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" Jun 07 09:13:33.566192 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:13:33.566172 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": container with ID starting with cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b not found: ID does not exist" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" Jun 07 09:13:33.566240 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.566202 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b"} err="failed to get container status \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": rpc error: code = NotFound desc = could not find container \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": container with ID starting with cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b not found: ID does not exist" Jun 07 09:13:33.566240 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.566221 2578 scope.go:117] "RemoveContainer" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" Jun 07 09:13:33.566458 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:13:33.566441 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": container with ID starting with c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a not found: ID does not exist" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" Jun 07 09:13:33.566499 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.566465 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a"} err="failed to get container status \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": rpc error: code = NotFound desc = could not find container \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": container with ID starting with c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a not found: ID does not exist" Jun 07 09:13:33.566499 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.566482 2578 scope.go:117] "RemoveContainer" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" Jun 07 09:13:33.566735 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:13:33.566719 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": container with ID starting with 5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc not found: ID does not exist" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" Jun 07 09:13:33.566789 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.566737 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc"} err="failed to get container status \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": rpc error: code = NotFound desc = could not find container \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": container with ID starting with 5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc not found: ID does not exist" Jun 07 09:13:33.566789 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.566750 2578 scope.go:117] "RemoveContainer" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" Jun 07 09:13:33.566992 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:13:33.566977 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": container with ID starting with 95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd not found: ID does not exist" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" Jun 07 09:13:33.567056 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.566994 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd"} err="failed to get container status \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": rpc error: code = NotFound desc = could not find container \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": container with ID starting with 95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd not found: ID does not exist" Jun 07 09:13:33.567056 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.567008 2578 scope.go:117] "RemoveContainer" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" Jun 07 09:13:33.567216 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:13:33.567198 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": container with ID starting with e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3 not found: ID does not exist" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" Jun 07 09:13:33.567254 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.567221 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3"} err="failed to get container status \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": rpc error: code = NotFound desc = could not find container \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": container with ID starting with e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3 not found: ID does not exist" Jun 07 09:13:33.567254 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.567237 2578 scope.go:117] "RemoveContainer" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" Jun 07 09:13:33.567476 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:13:33.567461 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": container with ID starting with 7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa not found: ID does not exist" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" Jun 07 09:13:33.567538 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.567479 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa"} err="failed to get container status \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": rpc error: code = NotFound desc = could not find container \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": container with ID starting with 7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa not found: ID does not exist" Jun 07 09:13:33.567538 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.567500 2578 scope.go:117] "RemoveContainer" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" Jun 07 09:13:33.567784 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:13:33.567745 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": container with ID starting with 822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26 not found: ID does not exist" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" Jun 07 09:13:33.567784 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.567765 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26"} err="failed to get container status \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": rpc error: code = NotFound desc = could not find container \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": container with ID starting with 822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26 not found: ID does not exist" Jun 07 09:13:33.567784 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.567782 2578 scope.go:117] "RemoveContainer" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" Jun 07 09:13:33.568198 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.568099 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b"} err="failed to get container status \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": rpc error: code = NotFound desc = could not find container \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": container with ID starting with cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b not found: ID does not exist" Jun 07 09:13:33.568198 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.568122 2578 scope.go:117] "RemoveContainer" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" Jun 07 09:13:33.568412 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.568393 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a"} err="failed to get container status \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": rpc error: code = NotFound desc = could not find container \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": container with ID starting with c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a not found: ID does not exist" Jun 07 09:13:33.568451 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.568413 2578 scope.go:117] "RemoveContainer" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" Jun 07 09:13:33.568672 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.568658 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc"} err="failed to get container status \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": rpc error: code = NotFound desc = could not find container \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": container with ID starting with 5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc not found: ID does not exist" Jun 07 09:13:33.568732 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.568674 2578 scope.go:117] "RemoveContainer" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" Jun 07 09:13:33.568918 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.568900 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd"} err="failed to get container status \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": rpc error: code = NotFound desc = could not find container \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": container with ID starting with 95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd not found: ID does not exist" Jun 07 09:13:33.568918 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.568934 2578 scope.go:117] "RemoveContainer" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" Jun 07 09:13:33.569184 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.569167 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3"} err="failed to get container status \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": rpc error: code = NotFound desc = could not find container \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": container with ID starting with e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3 not found: ID does not exist" Jun 07 09:13:33.569249 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.569185 2578 scope.go:117] "RemoveContainer" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" Jun 07 09:13:33.569401 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.569384 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa"} err="failed to get container status \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": rpc error: code = NotFound desc = could not find container \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": container with ID starting with 7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa not found: ID does not exist" Jun 07 09:13:33.569450 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.569402 2578 scope.go:117] "RemoveContainer" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" Jun 07 09:13:33.569587 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.569571 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26"} err="failed to get container status \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": rpc error: code = NotFound desc = could not find container \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": container with ID starting with 822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26 not found: ID does not exist" Jun 07 09:13:33.569631 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.569587 2578 scope.go:117] "RemoveContainer" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" Jun 07 09:13:33.569818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.569791 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b"} err="failed to get container status \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": rpc error: code = NotFound desc = could not find container \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": container with ID starting with cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b not found: ID does not exist" Jun 07 09:13:33.569818 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.569816 2578 scope.go:117] "RemoveContainer" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" Jun 07 09:13:33.570066 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570048 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a"} err="failed to get container status \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": rpc error: code = NotFound desc = could not find container \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": container with ID starting with c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a not found: ID does not exist" Jun 07 09:13:33.570126 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570068 2578 scope.go:117] "RemoveContainer" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" Jun 07 09:13:33.570278 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570262 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc"} err="failed to get container status \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": rpc error: code = NotFound desc = could not find container \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": container with ID starting with 5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc not found: ID does not exist" Jun 07 09:13:33.570278 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570277 2578 scope.go:117] "RemoveContainer" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" Jun 07 09:13:33.570483 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570464 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd"} err="failed to get container status \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": rpc error: code = NotFound desc = could not find container \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": container with ID starting with 95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd not found: ID does not exist" Jun 07 09:13:33.570533 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570484 2578 scope.go:117] "RemoveContainer" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" Jun 07 09:13:33.570685 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570665 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3"} err="failed to get container status \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": rpc error: code = NotFound desc = could not find container \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": container with ID starting with e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3 not found: ID does not exist" Jun 07 09:13:33.570733 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570687 2578 scope.go:117] "RemoveContainer" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" Jun 07 09:13:33.570900 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570886 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa"} err="failed to get container status \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": rpc error: code = NotFound desc = could not find container \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": container with ID starting with 7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa not found: ID does not exist" Jun 07 09:13:33.570900 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.570899 2578 scope.go:117] "RemoveContainer" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" Jun 07 09:13:33.571234 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.571215 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26"} err="failed to get container status \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": rpc error: code = NotFound desc = could not find container \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": container with ID starting with 822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26 not found: ID does not exist" Jun 07 09:13:33.571234 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.571233 2578 scope.go:117] "RemoveContainer" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" Jun 07 09:13:33.571464 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.571446 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b"} err="failed to get container status \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": rpc error: code = NotFound desc = could not find container \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": container with ID starting with cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b not found: ID does not exist" Jun 07 09:13:33.571504 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.571465 2578 scope.go:117] "RemoveContainer" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" Jun 07 09:13:33.571661 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.571643 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a"} err="failed to get container status \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": rpc error: code = NotFound desc = could not find container \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": container with ID starting with c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a not found: ID does not exist" Jun 07 09:13:33.571699 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.571661 2578 scope.go:117] "RemoveContainer" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" Jun 07 09:13:33.571843 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.571829 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc"} err="failed to get container status \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": rpc error: code = NotFound desc = could not find container \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": container with ID starting with 5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc not found: ID does not exist" Jun 07 09:13:33.571885 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.571842 2578 scope.go:117] "RemoveContainer" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" Jun 07 09:13:33.572120 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572101 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd"} err="failed to get container status \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": rpc error: code = NotFound desc = could not find container \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": container with ID starting with 95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd not found: ID does not exist" Jun 07 09:13:33.572120 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572120 2578 scope.go:117] "RemoveContainer" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" Jun 07 09:13:33.572336 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572317 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3"} err="failed to get container status \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": rpc error: code = NotFound desc = could not find container \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": container with ID starting with e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3 not found: ID does not exist" Jun 07 09:13:33.572379 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572336 2578 scope.go:117] "RemoveContainer" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" Jun 07 09:13:33.572561 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572536 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa"} err="failed to get container status \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": rpc error: code = NotFound desc = could not find container \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": container with ID starting with 7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa not found: ID does not exist" Jun 07 09:13:33.572561 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572560 2578 scope.go:117] "RemoveContainer" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" Jun 07 09:13:33.572755 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572740 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26"} err="failed to get container status \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": rpc error: code = NotFound desc = could not find container \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": container with ID starting with 822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26 not found: ID does not exist" Jun 07 09:13:33.572755 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572755 2578 scope.go:117] "RemoveContainer" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" Jun 07 09:13:33.572968 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572950 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b"} err="failed to get container status \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": rpc error: code = NotFound desc = could not find container \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": container with ID starting with cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b not found: ID does not exist" Jun 07 09:13:33.573039 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.572969 2578 scope.go:117] "RemoveContainer" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" Jun 07 09:13:33.573166 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573151 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a"} err="failed to get container status \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": rpc error: code = NotFound desc = could not find container \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": container with ID starting with c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a not found: ID does not exist" Jun 07 09:13:33.573205 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573166 2578 scope.go:117] "RemoveContainer" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" Jun 07 09:13:33.573352 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573337 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc"} err="failed to get container status \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": rpc error: code = NotFound desc = could not find container \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": container with ID starting with 5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc not found: ID does not exist" Jun 07 09:13:33.573352 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573351 2578 scope.go:117] "RemoveContainer" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" Jun 07 09:13:33.573511 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573490 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd"} err="failed to get container status \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": rpc error: code = NotFound desc = could not find container \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": container with ID starting with 95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd not found: ID does not exist" Jun 07 09:13:33.573511 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573506 2578 scope.go:117] "RemoveContainer" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" Jun 07 09:13:33.573718 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573701 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3"} err="failed to get container status \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": rpc error: code = NotFound desc = could not find container \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": container with ID starting with e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3 not found: ID does not exist" Jun 07 09:13:33.573765 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573719 2578 scope.go:117] "RemoveContainer" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" Jun 07 09:13:33.573958 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573915 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa"} err="failed to get container status \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": rpc error: code = NotFound desc = could not find container \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": container with ID starting with 7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa not found: ID does not exist" Jun 07 09:13:33.574018 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.573959 2578 scope.go:117] "RemoveContainer" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" Jun 07 09:13:33.574180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.574163 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26"} err="failed to get container status \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": rpc error: code = NotFound desc = could not find container \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": container with ID starting with 822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26 not found: ID does not exist" Jun 07 09:13:33.574219 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.574180 2578 scope.go:117] "RemoveContainer" containerID="cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b" Jun 07 09:13:33.574394 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.574371 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b"} err="failed to get container status \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": rpc error: code = NotFound desc = could not find container \"cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b\": container with ID starting with cac8124fbb78fdd3267d30a907279de532760575075dcb8f60e017ef1ee28c5b not found: ID does not exist" Jun 07 09:13:33.574477 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.574396 2578 scope.go:117] "RemoveContainer" containerID="c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a" Jun 07 09:13:33.574648 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.574630 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a"} err="failed to get container status \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": rpc error: code = NotFound desc = could not find container \"c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a\": container with ID starting with c755221ae7d2450e0ec69be80106498959a0a266b39ff515f436bd3745608c7a not found: ID does not exist" Jun 07 09:13:33.574688 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.574650 2578 scope.go:117] "RemoveContainer" containerID="5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc" Jun 07 09:13:33.574860 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.574843 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc"} err="failed to get container status \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": rpc error: code = NotFound desc = could not find container \"5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc\": container with ID starting with 5433c2f34223333a191e1a298f16bd0b784e1d3b248b691867d13660f3fcc7fc not found: ID does not exist" Jun 07 09:13:33.574904 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.574861 2578 scope.go:117] "RemoveContainer" containerID="95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd" Jun 07 09:13:33.575055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.575040 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd"} err="failed to get container status \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": rpc error: code = NotFound desc = could not find container \"95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd\": container with ID starting with 95de825a6db4f54a7ddfbe16eeb8f71b9b32014b868ad9f366ea413e5211aedd not found: ID does not exist" Jun 07 09:13:33.575098 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.575058 2578 scope.go:117] "RemoveContainer" containerID="e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3" Jun 07 09:13:33.575260 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.575220 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3"} err="failed to get container status \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": rpc error: code = NotFound desc = could not find container \"e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3\": container with ID starting with e0b8f50ac790e2def8c462a1b9f40354a066d8c7420ec662b59573dd50cc8fd3 not found: ID does not exist" Jun 07 09:13:33.575297 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.575263 2578 scope.go:117] "RemoveContainer" containerID="7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa" Jun 07 09:13:33.575451 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.575435 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa"} err="failed to get container status \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": rpc error: code = NotFound desc = could not find container \"7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa\": container with ID starting with 7c0e7fd5e9d7e856ab7777d2bc3f8b08fdfdb61d3c9bde903fadf279ad0e2dfa not found: ID does not exist" Jun 07 09:13:33.575497 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.575451 2578 scope.go:117] "RemoveContainer" containerID="822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26" Jun 07 09:13:33.575632 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.575610 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26"} err="failed to get container status \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": rpc error: code = NotFound desc = could not find container \"822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26\": container with ID starting with 822c6a8d14dccc8a4b2cf0b572e3794889a5a775c79adbda57579ba8d3aecc26 not found: ID does not exist" Jun 07 09:13:33.592953 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.592912 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzkm6\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-kube-api-access-bzkm6\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593073 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.592990 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config-out\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593073 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593012 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-serving-certs-ca-bundle\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593073 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593039 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-tls-assets\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593073 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593063 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-0\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593276 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593099 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-1\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593276 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593132 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-tls\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593276 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593162 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-2\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593276 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593194 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-grpc-tls\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593276 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593228 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593276 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593266 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-thanos-prometheus-http-client-file\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593291 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593325 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-kube-rbac-proxy\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593365 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-trusted-ca-bundle\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593393 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-metrics-client-certs\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593428 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-kubelet-serving-ca-bundle\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593453 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-db\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593464 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593484 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-web-config\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593508 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-metrics-client-ca\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.593560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593536 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\" (UID: \"8b31932c-4e6c-49c0-847f-4fbc6dd4a477\") " Jun 07 09:13:33.594503 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593748 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:33.594503 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593856 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.594503 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.593876 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.594503 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.594211 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:33.594725 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.594501 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:33.595773 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.595342 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:33.595773 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.595578 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:13:33.595773 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.595634 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:33.595999 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.595912 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:33.596246 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.596178 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.596332 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.596238 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config-out" (OuterVolumeSpecName: "config-out") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:13:33.596454 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.596429 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-kube-api-access-bzkm6" (OuterVolumeSpecName: "kube-api-access-bzkm6") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "kube-api-access-bzkm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:13:33.596613 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.596575 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.596713 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.596646 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:13:33.597062 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.597036 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.597888 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.597855 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.598029 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.598004 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.598165 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.598146 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config" (OuterVolumeSpecName: "config") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.598290 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.598268 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.598509 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.598493 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.609345 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.609323 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-web-config" (OuterVolumeSpecName: "web-config") pod "8b31932c-4e6c-49c0-847f-4fbc6dd4a477" (UID: "8b31932c-4e6c-49c0-847f-4fbc6dd4a477"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695010 2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-metrics-client-certs\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695060 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695071 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-db\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695080 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-web-config\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695089 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-configmap-metrics-client-ca\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695098 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695107 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzkm6\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-kube-api-access-bzkm6\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695115 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config-out\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695123 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-tls-assets\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695131 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695124 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695140 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695148 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695157 2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-grpc-tls\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695165 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695173 2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695182 2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-config\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695189 2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-secret-kube-rbac-proxy\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.695577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.695198 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b31932c-4e6c-49c0-847f-4fbc6dd4a477-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:33.837151 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.837118 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:13:33.843601 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.843573 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:13:33.867507 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867470 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:13:33.867866 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867850 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867870 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867888 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="prometheus" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867897 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="prometheus" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867916 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy-web" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867940 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy-web" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867952 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy-thanos" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867962 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy-thanos" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867974 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="config-reloader" Jun 07 09:13:33.867985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867982 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="config-reloader" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.867994 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="thanos-sidecar" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868001 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="thanos-sidecar" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868012 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="init-config-reloader" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868021 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="init-config-reloader" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868033 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49f4b2cb-9a9a-4b77-9868-2351008715b8" containerName="console" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868041 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="49f4b2cb-9a9a-4b77-9868-2351008715b8" containerName="console" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868124 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="49f4b2cb-9a9a-4b77-9868-2351008715b8" containerName="console" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868139 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="config-reloader" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868153 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="prometheus" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868163 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="thanos-sidecar" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868187 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy-web" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868207 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy-thanos" Jun 07 09:13:33.868409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.868216 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" containerName="kube-rbac-proxy" Jun 07 09:13:33.873783 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.873755 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.876401 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.876365 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 07 09:13:33.876401 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.876383 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 07 09:13:33.876650 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.876619 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xq7d8\"" Jun 07 09:13:33.876827 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.876807 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 07 09:13:33.876904 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.876881 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 07 09:13:33.877101 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877078 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 07 09:13:33.877193 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877108 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 07 09:13:33.877249 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877214 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 07 09:13:33.877361 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877341 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 07 09:13:33.877488 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877387 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 07 09:13:33.877635 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877619 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 07 09:13:33.877684 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877634 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 07 09:13:33.877845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877822 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 07 09:13:33.877845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877835 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 07 09:13:33.878016 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.877961 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9n9cn2nsip9a1\"" Jun 07 09:13:33.879815 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.879793 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 07 09:13:33.887382 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.885479 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 07 09:13:33.891026 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.890998 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:13:33.997699 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-config\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.997699 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/440b2bb3-6007-4795-9f02-5ec72cc97922-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.997972 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997728 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.997972 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.997972 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997830 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.997972 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997857 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgn95\" (UniqueName: \"kubernetes.io/projected/440b2bb3-6007-4795-9f02-5ec72cc97922-kube-api-access-tgn95\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.997972 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997875 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.997972 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.997972 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997918 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.997981 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998002 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998097 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/440b2bb3-6007-4795-9f02-5ec72cc97922-config-out\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998140 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998156 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-web-config\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998193 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:33.998244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:33.998226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099280 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099280 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/440b2bb3-6007-4795-9f02-5ec72cc97922-config-out\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099363 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-web-config\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099410 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-config\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.099515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/440b2bb3-6007-4795-9f02-5ec72cc97922-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgn95\" (UniqueName: \"kubernetes.io/projected/440b2bb3-6007-4795-9f02-5ec72cc97922-kube-api-access-tgn95\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100055 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.099727 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100452 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.100087 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.100452 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.100414 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.102108 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.102078 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.102269 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.102135 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.102269 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.102144 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.103339 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.102770 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.103339 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.102880 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.103339 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.103299 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.103556 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.103336 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.103556 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.103366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.103702 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.103680 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-web-config\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.104107 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.104085 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/440b2bb3-6007-4795-9f02-5ec72cc97922-config-out\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.104183 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.104103 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-config\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.104490 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.104473 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.104607 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.104591 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/440b2bb3-6007-4795-9f02-5ec72cc97922-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.104707 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.104686 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/440b2bb3-6007-4795-9f02-5ec72cc97922-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.105257 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.105239 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.105802 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.105787 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/440b2bb3-6007-4795-9f02-5ec72cc97922-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.108566 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.108548 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgn95\" (UniqueName: \"kubernetes.io/projected/440b2bb3-6007-4795-9f02-5ec72cc97922-kube-api-access-tgn95\") pod \"prometheus-k8s-0\" (UID: \"440b2bb3-6007-4795-9f02-5ec72cc97922\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.191034 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.191004 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:34.299189 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.299158 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b31932c-4e6c-49c0-847f-4fbc6dd4a477" path="/var/lib/kubelet/pods/8b31932c-4e6c-49c0-847f-4fbc6dd4a477/volumes" Jun 07 09:13:34.328064 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.328040 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 07 09:13:34.330175 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:13:34.330152 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440b2bb3_6007_4795_9f02_5ec72cc97922.slice/crio-c900ad450b552cc742dd2e94c0d4e63ff6c3037829f9be8a081a1d5092f69506 WatchSource:0}: Error finding container c900ad450b552cc742dd2e94c0d4e63ff6c3037829f9be8a081a1d5092f69506: Status 404 returned error can't find the container with id c900ad450b552cc742dd2e94c0d4e63ff6c3037829f9be8a081a1d5092f69506 Jun 07 09:13:34.517632 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.517600 2578 generic.go:356] "Generic (PLEG): container finished" podID="440b2bb3-6007-4795-9f02-5ec72cc97922" containerID="3efb30916e5b6f0b7c7c4012386c8293bd2d42f5bf766bc2e33c5b4515623ed2" exitCode=0 Jun 07 09:13:34.517822 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.517659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"440b2bb3-6007-4795-9f02-5ec72cc97922","Type":"ContainerDied","Data":"3efb30916e5b6f0b7c7c4012386c8293bd2d42f5bf766bc2e33c5b4515623ed2"} Jun 07 09:13:34.517822 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:34.517711 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"440b2bb3-6007-4795-9f02-5ec72cc97922","Type":"ContainerStarted","Data":"c900ad450b552cc742dd2e94c0d4e63ff6c3037829f9be8a081a1d5092f69506"} Jun 07 09:13:35.524794 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:35.524752 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"440b2bb3-6007-4795-9f02-5ec72cc97922","Type":"ContainerStarted","Data":"fc7684c257c067115e52a1ca91c487c632ab31293dce379294e213abb7f75970"} Jun 07 09:13:35.525176 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:35.524819 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"440b2bb3-6007-4795-9f02-5ec72cc97922","Type":"ContainerStarted","Data":"0b44d9e5dad64f3b23b8310627b3cd8806e8022b6a2030f79fb227784f5ea751"} Jun 07 09:13:35.525176 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:35.524850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"440b2bb3-6007-4795-9f02-5ec72cc97922","Type":"ContainerStarted","Data":"1c1d33772305c3de8dee2c80c81f07b76521f6a8b33abd4c9940b4d6b6b76140"} Jun 07 09:13:35.525176 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:35.524882 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"440b2bb3-6007-4795-9f02-5ec72cc97922","Type":"ContainerStarted","Data":"f8ce04af89bcc5f63cf5ae02c8797fa4810f5bb438ff82ffca713493719f0536"} Jun 07 09:13:35.525176 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:35.524911 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"440b2bb3-6007-4795-9f02-5ec72cc97922","Type":"ContainerStarted","Data":"65f51305c05acae5c6eb39211c49c7ddc7e56a41377a2b43525416605f7bed48"} Jun 07 09:13:35.525176 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:35.524974 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"440b2bb3-6007-4795-9f02-5ec72cc97922","Type":"ContainerStarted","Data":"36e5be4ab0b43be6a14b695722e9ff23597682775424cd77f74de2a27567443c"} Jun 07 09:13:35.555499 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:35.555447 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.535943489 podStartE2EDuration="2.555427816s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.484327ms" isStatefulPod=false podCreationTimestamp="2026-06-07 09:13:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:13:35.552535479 +0000 UTC m=+327.697596412" watchObservedRunningTime="2026-06-07 09:13:35.555427816 +0000 UTC m=+327.700488736" Jun 07 09:13:39.192016 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:39.191972 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:13:47.361901 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.361822 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" podUID="3c21300a-c19c-4af6-aaa0-f4927661a250" containerName="registry" containerID="cri-o://7b914a025f60111cccfdeeee5315a6bae1fbd381e78c2ec2b9e4c0b271a7ac33" gracePeriod=30 Jun 07 09:13:47.578548 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.578514 2578 generic.go:356] "Generic (PLEG): container finished" podID="3c21300a-c19c-4af6-aaa0-f4927661a250" containerID="7b914a025f60111cccfdeeee5315a6bae1fbd381e78c2ec2b9e4c0b271a7ac33" exitCode=0 Jun 07 09:13:47.578717 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.578570 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" event={"ID":"3c21300a-c19c-4af6-aaa0-f4927661a250","Type":"ContainerDied","Data":"7b914a025f60111cccfdeeee5315a6bae1fbd381e78c2ec2b9e4c0b271a7ac33"} Jun 07 09:13:47.610089 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.610061 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:13:47.722266 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722178 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-bound-sa-token\") pod \"3c21300a-c19c-4af6-aaa0-f4927661a250\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " Jun 07 09:13:47.722266 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722226 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-certificates\") pod \"3c21300a-c19c-4af6-aaa0-f4927661a250\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " Jun 07 09:13:47.722492 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722277 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-image-registry-private-configuration\") pod \"3c21300a-c19c-4af6-aaa0-f4927661a250\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " Jun 07 09:13:47.722492 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722313 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4rrk\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-kube-api-access-l4rrk\") pod \"3c21300a-c19c-4af6-aaa0-f4927661a250\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " Jun 07 09:13:47.722492 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722353 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-trusted-ca\") pod \"3c21300a-c19c-4af6-aaa0-f4927661a250\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " Jun 07 09:13:47.722492 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722392 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c21300a-c19c-4af6-aaa0-f4927661a250-ca-trust-extracted\") pod \"3c21300a-c19c-4af6-aaa0-f4927661a250\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " Jun 07 09:13:47.722492 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722441 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") pod \"3c21300a-c19c-4af6-aaa0-f4927661a250\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " Jun 07 09:13:47.722492 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722491 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-installation-pull-secrets\") pod \"3c21300a-c19c-4af6-aaa0-f4927661a250\" (UID: \"3c21300a-c19c-4af6-aaa0-f4927661a250\") " Jun 07 09:13:47.722797 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722733 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3c21300a-c19c-4af6-aaa0-f4927661a250" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:47.722920 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.722887 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3c21300a-c19c-4af6-aaa0-f4927661a250" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:13:47.725132 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.724994 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3c21300a-c19c-4af6-aaa0-f4927661a250" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:47.725238 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.725176 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3c21300a-c19c-4af6-aaa0-f4927661a250" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:13:47.725238 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.725208 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3c21300a-c19c-4af6-aaa0-f4927661a250" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:13:47.725418 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.725392 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3c21300a-c19c-4af6-aaa0-f4927661a250" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:13:47.725519 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.725444 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-kube-api-access-l4rrk" (OuterVolumeSpecName: "kube-api-access-l4rrk") pod "3c21300a-c19c-4af6-aaa0-f4927661a250" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250"). InnerVolumeSpecName "kube-api-access-l4rrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:13:47.732974 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.732919 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c21300a-c19c-4af6-aaa0-f4927661a250-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3c21300a-c19c-4af6-aaa0-f4927661a250" (UID: "3c21300a-c19c-4af6-aaa0-f4927661a250"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:13:47.823341 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.823301 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4rrk\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-kube-api-access-l4rrk\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:47.823341 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.823337 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-trusted-ca\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:47.823341 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.823350 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c21300a-c19c-4af6-aaa0-f4927661a250-ca-trust-extracted\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:47.823584 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.823362 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-tls\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:47.823584 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.823373 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-installation-pull-secrets\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:47.823584 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.823384 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c21300a-c19c-4af6-aaa0-f4927661a250-bound-sa-token\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:47.823584 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.823395 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c21300a-c19c-4af6-aaa0-f4927661a250-registry-certificates\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:47.823584 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:47.823407 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c21300a-c19c-4af6-aaa0-f4927661a250-image-registry-private-configuration\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:13:48.583958 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:48.583863 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" Jun 07 09:13:48.583958 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:48.583902 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c97d5c65c-w8rrw" event={"ID":"3c21300a-c19c-4af6-aaa0-f4927661a250","Type":"ContainerDied","Data":"a5437b4c4082c66395880dac2f60eced58602b9740787220ba194b89907f8d83"} Jun 07 09:13:48.584443 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:48.583977 2578 scope.go:117] "RemoveContainer" containerID="7b914a025f60111cccfdeeee5315a6bae1fbd381e78c2ec2b9e4c0b271a7ac33" Jun 07 09:13:48.601825 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:48.601794 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c97d5c65c-w8rrw"] Jun 07 09:13:48.605676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:48.605646 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c97d5c65c-w8rrw"] Jun 07 09:13:50.299144 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:13:50.299111 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c21300a-c19c-4af6-aaa0-f4927661a250" path="/var/lib/kubelet/pods/3c21300a-c19c-4af6-aaa0-f4927661a250/volumes" Jun 07 09:14:34.191723 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:34.191688 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:14:34.207050 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:34.207026 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:14:34.785970 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:34.785941 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 07 09:14:54.509332 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.509303 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-2scwp"] Jun 07 09:14:54.509671 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.509652 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c21300a-c19c-4af6-aaa0-f4927661a250" containerName="registry" Jun 07 09:14:54.509671 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.509668 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c21300a-c19c-4af6-aaa0-f4927661a250" containerName="registry" Jun 07 09:14:54.509741 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.509717 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c21300a-c19c-4af6-aaa0-f4927661a250" containerName="registry" Jun 07 09:14:54.512950 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.512918 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-2scwp" Jun 07 09:14:54.515509 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.515489 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 07 09:14:54.516601 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.516580 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 07 09:14:54.516707 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.516580 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wrckz\"" Jun 07 09:14:54.522067 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.521919 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-2scwp"] Jun 07 09:14:54.584139 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.584110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ada6e5f-15ec-4528-b8a2-913523e1d94b-bound-sa-token\") pod \"cert-manager-59c77684cc-2scwp\" (UID: \"9ada6e5f-15ec-4528-b8a2-913523e1d94b\") " pod="cert-manager/cert-manager-59c77684cc-2scwp" Jun 07 09:14:54.584289 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.584183 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72vwq\" (UniqueName: \"kubernetes.io/projected/9ada6e5f-15ec-4528-b8a2-913523e1d94b-kube-api-access-72vwq\") pod \"cert-manager-59c77684cc-2scwp\" (UID: \"9ada6e5f-15ec-4528-b8a2-913523e1d94b\") " pod="cert-manager/cert-manager-59c77684cc-2scwp" Jun 07 09:14:54.685501 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.685454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72vwq\" (UniqueName: \"kubernetes.io/projected/9ada6e5f-15ec-4528-b8a2-913523e1d94b-kube-api-access-72vwq\") pod \"cert-manager-59c77684cc-2scwp\" (UID: \"9ada6e5f-15ec-4528-b8a2-913523e1d94b\") " pod="cert-manager/cert-manager-59c77684cc-2scwp" Jun 07 09:14:54.685501 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.685506 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ada6e5f-15ec-4528-b8a2-913523e1d94b-bound-sa-token\") pod \"cert-manager-59c77684cc-2scwp\" (UID: \"9ada6e5f-15ec-4528-b8a2-913523e1d94b\") " pod="cert-manager/cert-manager-59c77684cc-2scwp" Jun 07 09:14:54.693301 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.693274 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ada6e5f-15ec-4528-b8a2-913523e1d94b-bound-sa-token\") pod \"cert-manager-59c77684cc-2scwp\" (UID: \"9ada6e5f-15ec-4528-b8a2-913523e1d94b\") " pod="cert-manager/cert-manager-59c77684cc-2scwp" Jun 07 09:14:54.693565 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.693543 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72vwq\" (UniqueName: \"kubernetes.io/projected/9ada6e5f-15ec-4528-b8a2-913523e1d94b-kube-api-access-72vwq\") pod \"cert-manager-59c77684cc-2scwp\" (UID: \"9ada6e5f-15ec-4528-b8a2-913523e1d94b\") " pod="cert-manager/cert-manager-59c77684cc-2scwp" Jun 07 09:14:54.830422 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.830344 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-2scwp" Jun 07 09:14:54.956521 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.956496 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-2scwp"] Jun 07 09:14:54.959192 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:14:54.959159 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ada6e5f_15ec_4528_b8a2_913523e1d94b.slice/crio-dbb6d1a6e45be49456ff52a70212b34ab8a44837f8c40a6eac1373330c700b11 WatchSource:0}: Error finding container dbb6d1a6e45be49456ff52a70212b34ab8a44837f8c40a6eac1373330c700b11: Status 404 returned error can't find the container with id dbb6d1a6e45be49456ff52a70212b34ab8a44837f8c40a6eac1373330c700b11 Jun 07 09:14:54.961107 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:54.961090 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 07 09:14:55.857545 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:55.857507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-2scwp" event={"ID":"9ada6e5f-15ec-4528-b8a2-913523e1d94b","Type":"ContainerStarted","Data":"dbb6d1a6e45be49456ff52a70212b34ab8a44837f8c40a6eac1373330c700b11"} Jun 07 09:14:57.867199 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:14:57.867106 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-2scwp" event={"ID":"9ada6e5f-15ec-4528-b8a2-913523e1d94b","Type":"ContainerStarted","Data":"02a2c58002ba130e742cfad1b11bb0db7ef249ac31c1fe30e84d914f3ba4cd1f"} Jun 07 09:15:24.874420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.874314 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-2scwp" podStartSLOduration=28.295562871 podStartE2EDuration="30.87429512s" totalImagesPullingTime="2.578732249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:14:57.88176536 +0000 UTC m=+410.026826279" watchObservedRunningTime="2026-06-07 09:15:24.87429512 +0000 UTC m=+437.019356038" Jun 07 09:15:24.874912 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.874894 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m"] Jun 07 09:15:24.877311 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.877293 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:24.879802 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.879778 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 07 09:15:24.881311 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.881289 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:15:24.881460 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.881325 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w4pn9\"" Jun 07 09:15:24.881460 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.881362 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 07 09:15:24.881614 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.881404 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 07 09:15:24.881614 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.881443 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 07 09:15:24.887060 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.887042 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m"] Jun 07 09:15:24.945316 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.945285 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18c76e8d-33c8-442c-ac2a-dc83407c0278-manager-config\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:24.945512 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.945356 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18c76e8d-33c8-442c-ac2a-dc83407c0278-cert\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:24.945512 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.945422 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18c76e8d-33c8-442c-ac2a-dc83407c0278-metrics-cert\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:24.945512 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:24.945460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvlhr\" (UniqueName: \"kubernetes.io/projected/18c76e8d-33c8-442c-ac2a-dc83407c0278-kube-api-access-fvlhr\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.046017 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.045985 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18c76e8d-33c8-442c-ac2a-dc83407c0278-metrics-cert\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.046017 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.046019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvlhr\" (UniqueName: \"kubernetes.io/projected/18c76e8d-33c8-442c-ac2a-dc83407c0278-kube-api-access-fvlhr\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.046263 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.046040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18c76e8d-33c8-442c-ac2a-dc83407c0278-manager-config\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.046263 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.046103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18c76e8d-33c8-442c-ac2a-dc83407c0278-cert\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.046793 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.046770 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/18c76e8d-33c8-442c-ac2a-dc83407c0278-manager-config\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.048500 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.048480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18c76e8d-33c8-442c-ac2a-dc83407c0278-cert\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.048601 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.048517 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/18c76e8d-33c8-442c-ac2a-dc83407c0278-metrics-cert\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.055056 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.055031 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvlhr\" (UniqueName: \"kubernetes.io/projected/18c76e8d-33c8-442c-ac2a-dc83407c0278-kube-api-access-fvlhr\") pod \"lws-controller-manager-867d454bbd-2gc9m\" (UID: \"18c76e8d-33c8-442c-ac2a-dc83407c0278\") " pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.188085 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.187984 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:25.328112 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.328086 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m"] Jun 07 09:15:25.330416 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:15:25.330389 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c76e8d_33c8_442c_ac2a_dc83407c0278.slice/crio-0635794c2ffbcc6f2530b2094984a18f333183be2c82653a9f090c4c74b51ff8 WatchSource:0}: Error finding container 0635794c2ffbcc6f2530b2094984a18f333183be2c82653a9f090c4c74b51ff8: Status 404 returned error can't find the container with id 0635794c2ffbcc6f2530b2094984a18f333183be2c82653a9f090c4c74b51ff8 Jun 07 09:15:25.987982 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:25.987947 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" event={"ID":"18c76e8d-33c8-442c-ac2a-dc83407c0278","Type":"ContainerStarted","Data":"0635794c2ffbcc6f2530b2094984a18f333183be2c82653a9f090c4c74b51ff8"} Jun 07 09:15:27.999255 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:27.999214 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" event={"ID":"18c76e8d-33c8-442c-ac2a-dc83407c0278","Type":"ContainerStarted","Data":"51ae243ed040c228bf6a7f7dc23804b91701f00671e08ba4c6440f0bca30d2fc"} Jun 07 09:15:27.999676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:27.999344 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:28.017007 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:28.016962 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" podStartSLOduration=1.6493103649999998 podStartE2EDuration="4.016948469s" totalImagesPullingTime="2.367638104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:15:28.014962079 +0000 UTC m=+440.160023003" watchObservedRunningTime="2026-06-07 09:15:28.016948469 +0000 UTC m=+440.162009387" Jun 07 09:15:33.227041 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.227010 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77"] Jun 07 09:15:33.230686 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.230661 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.233789 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.233761 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gmnw6\"" Jun 07 09:15:33.233958 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.233896 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 07 09:15:33.234343 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.234317 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 07 09:15:33.234424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.234362 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 07 09:15:33.234424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.234368 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 07 09:15:33.248557 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.248531 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77"] Jun 07 09:15:33.318938 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.318892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/206da481-8d6c-42d7-aa37-60ad403bc84e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.319111 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.319018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/206da481-8d6c-42d7-aa37-60ad403bc84e-manifests\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.319180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.319119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/206da481-8d6c-42d7-aa37-60ad403bc84e-tmp\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.319180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.319159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/206da481-8d6c-42d7-aa37-60ad403bc84e-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.319262 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.319246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx48h\" (UniqueName: \"kubernetes.io/projected/206da481-8d6c-42d7-aa37-60ad403bc84e-kube-api-access-tx48h\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.420568 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.420530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/206da481-8d6c-42d7-aa37-60ad403bc84e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.420756 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.420588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/206da481-8d6c-42d7-aa37-60ad403bc84e-manifests\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.420756 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.420632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/206da481-8d6c-42d7-aa37-60ad403bc84e-tmp\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.420756 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.420666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/206da481-8d6c-42d7-aa37-60ad403bc84e-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.420756 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.420720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx48h\" (UniqueName: \"kubernetes.io/projected/206da481-8d6c-42d7-aa37-60ad403bc84e-kube-api-access-tx48h\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.421119 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.421091 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/206da481-8d6c-42d7-aa37-60ad403bc84e-manifests\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.422780 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.422755 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/206da481-8d6c-42d7-aa37-60ad403bc84e-tmp\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.423058 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.423039 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/206da481-8d6c-42d7-aa37-60ad403bc84e-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.423103 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.423082 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/206da481-8d6c-42d7-aa37-60ad403bc84e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.436380 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.436354 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx48h\" (UniqueName: \"kubernetes.io/projected/206da481-8d6c-42d7-aa37-60ad403bc84e-kube-api-access-tx48h\") pod \"opendatahub-operator-controller-manager-84c76964bf-xbx77\" (UID: \"206da481-8d6c-42d7-aa37-60ad403bc84e\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.542049 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.541955 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:33.683520 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:33.683495 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77"] Jun 07 09:15:33.685681 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:15:33.685658 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod206da481_8d6c_42d7_aa37_60ad403bc84e.slice/crio-96b56717dbd7b01792a1ed21a34dee5d9fabdd979282085ec1b08ae104700d67 WatchSource:0}: Error finding container 96b56717dbd7b01792a1ed21a34dee5d9fabdd979282085ec1b08ae104700d67: Status 404 returned error can't find the container with id 96b56717dbd7b01792a1ed21a34dee5d9fabdd979282085ec1b08ae104700d67 Jun 07 09:15:34.027129 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:34.027090 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" event={"ID":"206da481-8d6c-42d7-aa37-60ad403bc84e","Type":"ContainerStarted","Data":"96b56717dbd7b01792a1ed21a34dee5d9fabdd979282085ec1b08ae104700d67"} Jun 07 09:15:37.044141 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:37.044107 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" event={"ID":"206da481-8d6c-42d7-aa37-60ad403bc84e","Type":"ContainerStarted","Data":"e52432a350fb2de5c5ce565dda90627d41e23ac4dfca8e8f8ec52feb13b6e261"} Jun 07 09:15:38.050148 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:38.050112 2578 generic.go:356] "Generic (PLEG): container finished" podID="206da481-8d6c-42d7-aa37-60ad403bc84e" containerID="e52432a350fb2de5c5ce565dda90627d41e23ac4dfca8e8f8ec52feb13b6e261" exitCode=0 Jun 07 09:15:38.050563 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:38.050197 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" event={"ID":"206da481-8d6c-42d7-aa37-60ad403bc84e","Type":"ContainerDied","Data":"e52432a350fb2de5c5ce565dda90627d41e23ac4dfca8e8f8ec52feb13b6e261"} Jun 07 09:15:39.006409 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:39.006384 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-867d454bbd-2gc9m" Jun 07 09:15:39.057251 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:39.057218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" event={"ID":"206da481-8d6c-42d7-aa37-60ad403bc84e","Type":"ContainerStarted","Data":"481d79df86dbf9f8bac97fd15378496b1f9c37431243ed7bff5e49b4c11f3367"} Jun 07 09:15:39.057630 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:39.057389 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:39.078723 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:39.078678 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" podStartSLOduration=1.915228666 podStartE2EDuration="6.07866393s" totalImagesPullingTime="3.203087427s" totalInitContainerRuntime="960.347837ms" isStatefulPod=false podCreationTimestamp="2026-06-07 09:15:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:15:39.076757568 +0000 UTC m=+451.221818489" watchObservedRunningTime="2026-06-07 09:15:39.07866393 +0000 UTC m=+451.223724850" Jun 07 09:15:50.063966 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:50.063908 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-xbx77" Jun 07 09:15:52.360074 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.360040 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm"] Jun 07 09:15:52.489372 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.489333 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm"] Jun 07 09:15:52.489561 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.489425 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.492267 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.492238 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 07 09:15:52.492267 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.492245 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 07 09:15:52.492466 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.492275 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 07 09:15:52.492466 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.492249 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-flz7p\"" Jun 07 09:15:52.493416 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.493398 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 07 09:15:52.592692 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.592659 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/982eacff-74c5-443b-8c38-38c530a3aae2-tls-certs\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.592692 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.592693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982eacff-74c5-443b-8c38-38c530a3aae2-tmp\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.592888 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.592723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd6js\" (UniqueName: \"kubernetes.io/projected/982eacff-74c5-443b-8c38-38c530a3aae2-kube-api-access-hd6js\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.694170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.694085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/982eacff-74c5-443b-8c38-38c530a3aae2-tls-certs\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.694170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.694122 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982eacff-74c5-443b-8c38-38c530a3aae2-tmp\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.694170 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.694155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd6js\" (UniqueName: \"kubernetes.io/projected/982eacff-74c5-443b-8c38-38c530a3aae2-kube-api-access-hd6js\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.696406 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.696376 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982eacff-74c5-443b-8c38-38c530a3aae2-tmp\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.696670 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.696650 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/982eacff-74c5-443b-8c38-38c530a3aae2-tls-certs\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.702778 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.702756 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd6js\" (UniqueName: \"kubernetes.io/projected/982eacff-74c5-443b-8c38-38c530a3aae2-kube-api-access-hd6js\") pod \"kube-auth-proxy-f78cc86d8-vcrkm\" (UID: \"982eacff-74c5-443b-8c38-38c530a3aae2\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.799772 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.799739 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" Jun 07 09:15:52.951590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:52.951560 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm"] Jun 07 09:15:52.953343 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:15:52.953313 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982eacff_74c5_443b_8c38_38c530a3aae2.slice/crio-bce1aafb409b59e7cc8d5050178096b12e8fc910a63bc3a73f1734ee8adc12be WatchSource:0}: Error finding container bce1aafb409b59e7cc8d5050178096b12e8fc910a63bc3a73f1734ee8adc12be: Status 404 returned error can't find the container with id bce1aafb409b59e7cc8d5050178096b12e8fc910a63bc3a73f1734ee8adc12be Jun 07 09:15:53.128512 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:53.128445 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" event={"ID":"982eacff-74c5-443b-8c38-38c530a3aae2","Type":"ContainerStarted","Data":"bce1aafb409b59e7cc8d5050178096b12e8fc910a63bc3a73f1734ee8adc12be"} Jun 07 09:15:56.144686 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:56.144645 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" event={"ID":"982eacff-74c5-443b-8c38-38c530a3aae2","Type":"ContainerStarted","Data":"fbb6a7316987bdc7ca73373a6443b1b06ea93d1fe7cef229f256104e2d17b153"} Jun 07 09:15:56.162053 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:15:56.162002 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-vcrkm" podStartSLOduration=1.49790231 podStartE2EDuration="4.161987803s" totalImagesPullingTime="2.664085493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:15:56.160544652 +0000 UTC m=+468.305605797" watchObservedRunningTime="2026-06-07 09:15:56.161987803 +0000 UTC m=+468.307048775" Jun 07 09:17:36.897973 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:36.897937 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf"] Jun 07 09:17:36.901437 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:36.901415 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:36.904180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:36.904160 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-dvwkl\"" Jun 07 09:17:36.904281 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:36.904205 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 07 09:17:36.905468 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:36.905452 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 07 09:17:36.911891 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:36.911872 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf"] Jun 07 09:17:37.015220 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:37.015187 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ntkf\" (UniqueName: \"kubernetes.io/projected/c26140d9-de13-4a16-9803-f048f2f94002-kube-api-access-5ntkf\") pod \"limitador-operator-controller-manager-859d7b78bd-ldxgf\" (UID: \"c26140d9-de13-4a16-9803-f048f2f94002\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:37.116033 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:37.115998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ntkf\" (UniqueName: \"kubernetes.io/projected/c26140d9-de13-4a16-9803-f048f2f94002-kube-api-access-5ntkf\") pod \"limitador-operator-controller-manager-859d7b78bd-ldxgf\" (UID: \"c26140d9-de13-4a16-9803-f048f2f94002\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:37.124323 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:37.124292 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ntkf\" (UniqueName: \"kubernetes.io/projected/c26140d9-de13-4a16-9803-f048f2f94002-kube-api-access-5ntkf\") pod \"limitador-operator-controller-manager-859d7b78bd-ldxgf\" (UID: \"c26140d9-de13-4a16-9803-f048f2f94002\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:37.212499 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:37.212426 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:37.338169 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:37.338141 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf"] Jun 07 09:17:37.339640 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:17:37.339611 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26140d9_de13_4a16_9803_f048f2f94002.slice/crio-1bb78c7fa74685362e753f2f864f7d2979c99f893ffeed189ad5e28729905dc3 WatchSource:0}: Error finding container 1bb78c7fa74685362e753f2f864f7d2979c99f893ffeed189ad5e28729905dc3: Status 404 returned error can't find the container with id 1bb78c7fa74685362e753f2f864f7d2979c99f893ffeed189ad5e28729905dc3 Jun 07 09:17:37.610540 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:37.610504 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" event={"ID":"c26140d9-de13-4a16-9803-f048f2f94002","Type":"ContainerStarted","Data":"1bb78c7fa74685362e753f2f864f7d2979c99f893ffeed189ad5e28729905dc3"} Jun 07 09:17:39.621283 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:39.621254 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" event={"ID":"c26140d9-de13-4a16-9803-f048f2f94002","Type":"ContainerStarted","Data":"423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237"} Jun 07 09:17:39.621670 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:39.621314 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:39.643395 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:39.643351 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" podStartSLOduration=1.45920019 podStartE2EDuration="3.643336711s" totalImagesPullingTime="2.184136521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:17:39.642277931 +0000 UTC m=+571.787338855" watchObservedRunningTime="2026-06-07 09:17:39.643336711 +0000 UTC m=+571.788397630" Jun 07 09:17:47.931151 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:47.931115 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf"] Jun 07 09:17:47.931953 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:47.931885 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" podUID="c26140d9-de13-4a16-9803-f048f2f94002" containerName="manager" containerID="cri-o://423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237" gracePeriod=2 Jun 07 09:17:47.933737 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:47.933714 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:47.948540 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:47.948515 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf"] Jun 07 09:17:47.956092 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:17:47.956013 2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c26140d9-de13-4a16-9803-f048f2f94002\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-ldxgf\": pods \"limitador-operator-controller-manager-859d7b78bd-ldxgf\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:47.958324 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:17:47.958292 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldxgf\" is forbidden: User \"system:node:ip-10-0-129-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-203.ec2.internal' and this object" podUID="c26140d9-de13-4a16-9803-f048f2f94002" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:48.176594 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.176572 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:48.179069 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:17:48.179032 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldxgf\" is forbidden: User \"system:node:ip-10-0-129-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-203.ec2.internal' and this object" podUID="c26140d9-de13-4a16-9803-f048f2f94002" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:48.211731 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.211676 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ntkf\" (UniqueName: \"kubernetes.io/projected/c26140d9-de13-4a16-9803-f048f2f94002-kube-api-access-5ntkf\") pod \"c26140d9-de13-4a16-9803-f048f2f94002\" (UID: \"c26140d9-de13-4a16-9803-f048f2f94002\") " Jun 07 09:17:48.213663 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.213636 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c26140d9-de13-4a16-9803-f048f2f94002-kube-api-access-5ntkf" (OuterVolumeSpecName: "kube-api-access-5ntkf") pod "c26140d9-de13-4a16-9803-f048f2f94002" (UID: "c26140d9-de13-4a16-9803-f048f2f94002"). InnerVolumeSpecName "kube-api-access-5ntkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:17:48.298941 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:17:48.298890 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-ldxgf\" is forbidden: User \"system:node:ip-10-0-129-203.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-203.ec2.internal' and this object" podUID="c26140d9-de13-4a16-9803-f048f2f94002" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:48.299649 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.299629 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c26140d9-de13-4a16-9803-f048f2f94002" path="/var/lib/kubelet/pods/c26140d9-de13-4a16-9803-f048f2f94002/volumes" Jun 07 09:17:48.312619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.312600 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ntkf\" (UniqueName: \"kubernetes.io/projected/c26140d9-de13-4a16-9803-f048f2f94002-kube-api-access-5ntkf\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:17:48.632998 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.632966 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk"] Jun 07 09:17:48.633337 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.633326 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c26140d9-de13-4a16-9803-f048f2f94002" containerName="manager" Jun 07 09:17:48.633380 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.633339 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c26140d9-de13-4a16-9803-f048f2f94002" containerName="manager" Jun 07 09:17:48.633419 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.633409 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c26140d9-de13-4a16-9803-f048f2f94002" containerName="manager" Jun 07 09:17:48.636365 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.636349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:48.639206 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.639184 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4ljbr\"" Jun 07 09:17:48.651388 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.651363 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk"] Jun 07 09:17:48.664808 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.664782 2578 generic.go:356] "Generic (PLEG): container finished" podID="c26140d9-de13-4a16-9803-f048f2f94002" containerID="423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237" exitCode=0 Jun 07 09:17:48.664942 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.664884 2578 scope.go:117] "RemoveContainer" containerID="423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237" Jun 07 09:17:48.665017 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.664958 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldxgf" Jun 07 09:17:48.673452 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.673432 2578 scope.go:117] "RemoveContainer" containerID="423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237" Jun 07 09:17:48.673711 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:17:48.673691 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237\": container with ID starting with 423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237 not found: ID does not exist" containerID="423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237" Jun 07 09:17:48.673782 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.673723 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237"} err="failed to get container status \"423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237\": rpc error: code = NotFound desc = could not find container \"423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237\": container with ID starting with 423c51ed57f681acce67314e3fd14cb0985f6c5acfca52b0061ed60dbe5f8237 not found: ID does not exist" Jun 07 09:17:48.715227 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.715198 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp474\" (UniqueName: \"kubernetes.io/projected/3a9c326f-059c-4376-8b66-8004204491b1-kube-api-access-rp474\") pod \"kuadrant-operator-controller-manager-86878c769f-cl5rk\" (UID: \"3a9c326f-059c-4376-8b66-8004204491b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:48.715369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.715230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a9c326f-059c-4376-8b66-8004204491b1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cl5rk\" (UID: \"3a9c326f-059c-4376-8b66-8004204491b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:48.816205 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.816161 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp474\" (UniqueName: \"kubernetes.io/projected/3a9c326f-059c-4376-8b66-8004204491b1-kube-api-access-rp474\") pod \"kuadrant-operator-controller-manager-86878c769f-cl5rk\" (UID: \"3a9c326f-059c-4376-8b66-8004204491b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:48.816205 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.816209 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a9c326f-059c-4376-8b66-8004204491b1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cl5rk\" (UID: \"3a9c326f-059c-4376-8b66-8004204491b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:48.816663 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.816645 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a9c326f-059c-4376-8b66-8004204491b1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cl5rk\" (UID: \"3a9c326f-059c-4376-8b66-8004204491b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:48.825672 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.825653 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp474\" (UniqueName: \"kubernetes.io/projected/3a9c326f-059c-4376-8b66-8004204491b1-kube-api-access-rp474\") pod \"kuadrant-operator-controller-manager-86878c769f-cl5rk\" (UID: \"3a9c326f-059c-4376-8b66-8004204491b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:48.946445 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:48.946367 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:49.104796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:49.104773 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk"] Jun 07 09:17:49.107023 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:17:49.106992 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a9c326f_059c_4376_8b66_8004204491b1.slice/crio-ecb925e036c6ca124722bba01f1904a0330538a0d4725e7d815fd78c8fc77b2f WatchSource:0}: Error finding container ecb925e036c6ca124722bba01f1904a0330538a0d4725e7d815fd78c8fc77b2f: Status 404 returned error can't find the container with id ecb925e036c6ca124722bba01f1904a0330538a0d4725e7d815fd78c8fc77b2f Jun 07 09:17:49.674810 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:49.674771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" event={"ID":"3a9c326f-059c-4376-8b66-8004204491b1","Type":"ContainerStarted","Data":"ecb925e036c6ca124722bba01f1904a0330538a0d4725e7d815fd78c8fc77b2f"} Jun 07 09:17:52.691847 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:52.691761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" event={"ID":"3a9c326f-059c-4376-8b66-8004204491b1","Type":"ContainerStarted","Data":"a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1"} Jun 07 09:17:52.691847 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:52.691839 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:17:52.724637 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:17:52.724582 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" podStartSLOduration=1.598646376 podStartE2EDuration="4.724563988s" totalImagesPullingTime="3.125917612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:17:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:17:52.722468476 +0000 UTC m=+584.867529394" watchObservedRunningTime="2026-06-07 09:17:52.724563988 +0000 UTC m=+584.869624911" Jun 07 09:18:03.698764 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:03.698728 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:18:08.285557 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:08.285526 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:18:08.286083 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:08.285794 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:18:25.188176 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.188140 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mcssc"] Jun 07 09:18:25.190963 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.190919 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mcssc" Jun 07 09:18:25.193821 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.193606 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cbbml\"" Jun 07 09:18:25.195579 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.195556 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mcssc"] Jun 07 09:18:25.244537 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.244506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcxmq\" (UniqueName: \"kubernetes.io/projected/84c7ae72-e9c1-4238-b73b-214cfc865f5e-kube-api-access-hcxmq\") pod \"authorino-58cd955675-mcssc\" (UID: \"84c7ae72-e9c1-4238-b73b-214cfc865f5e\") " pod="kuadrant-system/authorino-58cd955675-mcssc" Jun 07 09:18:25.345029 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.344996 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcxmq\" (UniqueName: \"kubernetes.io/projected/84c7ae72-e9c1-4238-b73b-214cfc865f5e-kube-api-access-hcxmq\") pod \"authorino-58cd955675-mcssc\" (UID: \"84c7ae72-e9c1-4238-b73b-214cfc865f5e\") " pod="kuadrant-system/authorino-58cd955675-mcssc" Jun 07 09:18:25.353663 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.353641 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcxmq\" (UniqueName: \"kubernetes.io/projected/84c7ae72-e9c1-4238-b73b-214cfc865f5e-kube-api-access-hcxmq\") pod \"authorino-58cd955675-mcssc\" (UID: \"84c7ae72-e9c1-4238-b73b-214cfc865f5e\") " pod="kuadrant-system/authorino-58cd955675-mcssc" Jun 07 09:18:25.470966 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.470875 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-k7zt6"] Jun 07 09:18:25.473667 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.473643 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k7zt6" Jun 07 09:18:25.481243 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.481215 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k7zt6"] Jun 07 09:18:25.500655 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.500630 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mcssc" Jun 07 09:18:25.546488 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.546456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6lkh\" (UniqueName: \"kubernetes.io/projected/9dfb8592-ed72-480a-91bb-898703ec31e9-kube-api-access-x6lkh\") pod \"authorino-75d556bd77-k7zt6\" (UID: \"9dfb8592-ed72-480a-91bb-898703ec31e9\") " pod="kuadrant-system/authorino-75d556bd77-k7zt6" Jun 07 09:18:25.625113 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.625087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mcssc"] Jun 07 09:18:25.627351 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:18:25.627326 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84c7ae72_e9c1_4238_b73b_214cfc865f5e.slice/crio-7942fce9ac1beedf6d3f89a3529fc5dc5e9973a2688188409458b20c147dc076 WatchSource:0}: Error finding container 7942fce9ac1beedf6d3f89a3529fc5dc5e9973a2688188409458b20c147dc076: Status 404 returned error can't find the container with id 7942fce9ac1beedf6d3f89a3529fc5dc5e9973a2688188409458b20c147dc076 Jun 07 09:18:25.647065 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.647040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6lkh\" (UniqueName: \"kubernetes.io/projected/9dfb8592-ed72-480a-91bb-898703ec31e9-kube-api-access-x6lkh\") pod \"authorino-75d556bd77-k7zt6\" (UID: \"9dfb8592-ed72-480a-91bb-898703ec31e9\") " pod="kuadrant-system/authorino-75d556bd77-k7zt6" Jun 07 09:18:25.654549 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.654526 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6lkh\" (UniqueName: \"kubernetes.io/projected/9dfb8592-ed72-480a-91bb-898703ec31e9-kube-api-access-x6lkh\") pod \"authorino-75d556bd77-k7zt6\" (UID: \"9dfb8592-ed72-480a-91bb-898703ec31e9\") " pod="kuadrant-system/authorino-75d556bd77-k7zt6" Jun 07 09:18:25.783885 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.783847 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k7zt6" Jun 07 09:18:25.845935 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.845892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mcssc" event={"ID":"84c7ae72-e9c1-4238-b73b-214cfc865f5e","Type":"ContainerStarted","Data":"7942fce9ac1beedf6d3f89a3529fc5dc5e9973a2688188409458b20c147dc076"} Jun 07 09:18:25.910889 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:25.910859 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k7zt6"] Jun 07 09:18:25.912779 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:18:25.912752 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dfb8592_ed72_480a_91bb_898703ec31e9.slice/crio-46c9d71a6b241c0e7335c86accf0896a00e498bbec20f7da4756b2cbd4942294 WatchSource:0}: Error finding container 46c9d71a6b241c0e7335c86accf0896a00e498bbec20f7da4756b2cbd4942294: Status 404 returned error can't find the container with id 46c9d71a6b241c0e7335c86accf0896a00e498bbec20f7da4756b2cbd4942294 Jun 07 09:18:26.855240 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:26.855196 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k7zt6" event={"ID":"9dfb8592-ed72-480a-91bb-898703ec31e9","Type":"ContainerStarted","Data":"46c9d71a6b241c0e7335c86accf0896a00e498bbec20f7da4756b2cbd4942294"} Jun 07 09:18:29.875366 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:29.875328 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mcssc" event={"ID":"84c7ae72-e9c1-4238-b73b-214cfc865f5e","Type":"ContainerStarted","Data":"1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674"} Jun 07 09:18:29.876679 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:29.876658 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k7zt6" event={"ID":"9dfb8592-ed72-480a-91bb-898703ec31e9","Type":"ContainerStarted","Data":"ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d"} Jun 07 09:18:29.891205 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:29.891161 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mcssc" podStartSLOduration=1.2588327289999999 podStartE2EDuration="4.891145734s" totalImagesPullingTime="3.632313005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:18:29.889630604 +0000 UTC m=+622.034691536" watchObservedRunningTime="2026-06-07 09:18:29.891145734 +0000 UTC m=+622.036206663" Jun 07 09:18:29.904671 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:29.904619 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-k7zt6" podStartSLOduration=1.568404664 podStartE2EDuration="4.904603504s" totalImagesPullingTime="3.33619884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:18:29.903184096 +0000 UTC m=+622.048245015" watchObservedRunningTime="2026-06-07 09:18:29.904603504 +0000 UTC m=+622.049664425" Jun 07 09:18:29.927646 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:29.927615 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mcssc"] Jun 07 09:18:31.886166 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:31.886125 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mcssc" podUID="84c7ae72-e9c1-4238-b73b-214cfc865f5e" containerName="authorino" containerID="cri-o://1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674" gracePeriod=30 Jun 07 09:18:32.140098 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.140047 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mcssc" Jun 07 09:18:32.208498 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.208472 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcxmq\" (UniqueName: \"kubernetes.io/projected/84c7ae72-e9c1-4238-b73b-214cfc865f5e-kube-api-access-hcxmq\") pod \"84c7ae72-e9c1-4238-b73b-214cfc865f5e\" (UID: \"84c7ae72-e9c1-4238-b73b-214cfc865f5e\") " Jun 07 09:18:32.210427 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.210397 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84c7ae72-e9c1-4238-b73b-214cfc865f5e-kube-api-access-hcxmq" (OuterVolumeSpecName: "kube-api-access-hcxmq") pod "84c7ae72-e9c1-4238-b73b-214cfc865f5e" (UID: "84c7ae72-e9c1-4238-b73b-214cfc865f5e"). InnerVolumeSpecName "kube-api-access-hcxmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:18:32.309255 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.309233 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcxmq\" (UniqueName: \"kubernetes.io/projected/84c7ae72-e9c1-4238-b73b-214cfc865f5e-kube-api-access-hcxmq\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:18:32.892233 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.892198 2578 generic.go:356] "Generic (PLEG): container finished" podID="84c7ae72-e9c1-4238-b73b-214cfc865f5e" containerID="1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674" exitCode=0 Jun 07 09:18:32.892631 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.892254 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mcssc" Jun 07 09:18:32.892631 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.892259 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mcssc" event={"ID":"84c7ae72-e9c1-4238-b73b-214cfc865f5e","Type":"ContainerDied","Data":"1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674"} Jun 07 09:18:32.892631 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.892365 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mcssc" event={"ID":"84c7ae72-e9c1-4238-b73b-214cfc865f5e","Type":"ContainerDied","Data":"7942fce9ac1beedf6d3f89a3529fc5dc5e9973a2688188409458b20c147dc076"} Jun 07 09:18:32.892631 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.892387 2578 scope.go:117] "RemoveContainer" containerID="1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674" Jun 07 09:18:32.900779 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.900762 2578 scope.go:117] "RemoveContainer" containerID="1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674" Jun 07 09:18:32.901042 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:18:32.901025 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674\": container with ID starting with 1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674 not found: ID does not exist" containerID="1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674" Jun 07 09:18:32.901100 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.901054 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674"} err="failed to get container status \"1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674\": rpc error: code = NotFound desc = could not find container \"1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674\": container with ID starting with 1741b1274236f01d4d5d1ac3c40de1480940f9f2e4f976f155baaa0aef9c4674 not found: ID does not exist" Jun 07 09:18:32.908481 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.908459 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mcssc"] Jun 07 09:18:32.912424 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:32.912404 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mcssc"] Jun 07 09:18:34.299717 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:18:34.299684 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84c7ae72-e9c1-4238-b73b-214cfc865f5e" path="/var/lib/kubelet/pods/84c7ae72-e9c1-4238-b73b-214cfc865f5e/volumes" Jun 07 09:19:18.876661 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.876627 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:19:18.877159 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.877019 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84c7ae72-e9c1-4238-b73b-214cfc865f5e" containerName="authorino" Jun 07 09:19:18.877159 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.877032 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="84c7ae72-e9c1-4238-b73b-214cfc865f5e" containerName="authorino" Jun 07 09:19:18.877159 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.877109 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="84c7ae72-e9c1-4238-b73b-214cfc865f5e" containerName="authorino" Jun 07 09:19:18.881751 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.881730 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:18.884787 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.884767 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 07 09:19:18.884787 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.884771 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 07 09:19:18.884970 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.884815 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 07 09:19:18.886020 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.886001 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-vcxn7\"" Jun 07 09:19:18.892693 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.892671 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:19:18.907662 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:18.907638 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6g8h\" (UniqueName: \"kubernetes.io/projected/6c6eab1b-1adb-439e-bb19-85db8b5432e6-kube-api-access-n6g8h\") pod \"maas-keycloak-0\" (UID: \"6c6eab1b-1adb-439e-bb19-85db8b5432e6\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:19.008578 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:19.008541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6g8h\" (UniqueName: \"kubernetes.io/projected/6c6eab1b-1adb-439e-bb19-85db8b5432e6-kube-api-access-n6g8h\") pod \"maas-keycloak-0\" (UID: \"6c6eab1b-1adb-439e-bb19-85db8b5432e6\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:19.017327 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:19.017303 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6g8h\" (UniqueName: \"kubernetes.io/projected/6c6eab1b-1adb-439e-bb19-85db8b5432e6-kube-api-access-n6g8h\") pod \"maas-keycloak-0\" (UID: \"6c6eab1b-1adb-439e-bb19-85db8b5432e6\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:19.191616 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:19.191518 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:19.325920 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:19.325897 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:19:19.327659 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:19:19.327630 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6eab1b_1adb_439e_bb19_85db8b5432e6.slice/crio-36a5e9d4398574cd1c5eb3b18f069e2dff42321c44e97a3ffd3a90b5f783fa9d WatchSource:0}: Error finding container 36a5e9d4398574cd1c5eb3b18f069e2dff42321c44e97a3ffd3a90b5f783fa9d: Status 404 returned error can't find the container with id 36a5e9d4398574cd1c5eb3b18f069e2dff42321c44e97a3ffd3a90b5f783fa9d Jun 07 09:19:20.124032 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:20.123971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"6c6eab1b-1adb-439e-bb19-85db8b5432e6","Type":"ContainerStarted","Data":"36a5e9d4398574cd1c5eb3b18f069e2dff42321c44e97a3ffd3a90b5f783fa9d"} Jun 07 09:19:25.154199 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:25.154154 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"6c6eab1b-1adb-439e-bb19-85db8b5432e6","Type":"ContainerStarted","Data":"5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219"} Jun 07 09:19:25.172944 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:25.172879 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=2.057662747 podStartE2EDuration="7.172861632s" totalImagesPullingTime="5.115198885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:19:25.171585231 +0000 UTC m=+677.316646167" watchObservedRunningTime="2026-06-07 09:19:25.172861632 +0000 UTC m=+677.317922572" Jun 07 09:19:25.192712 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:25.192671 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:25.194183 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:25.194153 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:26.192136 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:26.192084 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:27.192310 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:27.192267 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:28.192568 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:28.192519 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:29.191894 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:29.191835 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:29.192765 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:29.192730 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:30.192632 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:30.192220 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:31.193163 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:31.193113 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:32.192686 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:32.192626 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:33.192264 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:33.192214 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:34.192282 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:34.192229 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:35.192319 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:35.192258 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:36.193103 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:36.193052 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:37.192246 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:37.192197 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.34:9000/health/started\": dial tcp 10.132.0.34:9000: connect: connection refused" Jun 07 09:19:38.309302 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:38.309263 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:38.324430 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:38.324371 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:19:48.317393 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:48.317366 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 07 09:19:49.198812 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.198776 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hm4gj"] Jun 07 09:19:49.207571 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.207533 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hm4gj" Jun 07 09:19:49.208452 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.208426 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hm4gj"] Jun 07 09:19:49.313392 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.313354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd5ml\" (UniqueName: \"kubernetes.io/projected/909b721e-40bc-46e6-888f-add43e477fc7-kube-api-access-dd5ml\") pod \"authorino-5d4b67c446-hm4gj\" (UID: \"909b721e-40bc-46e6-888f-add43e477fc7\") " pod="kuadrant-system/authorino-5d4b67c446-hm4gj" Jun 07 09:19:49.379953 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.379884 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hm4gj"] Jun 07 09:19:49.380397 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:19:49.380244 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-dd5ml], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-hm4gj" podUID="909b721e-40bc-46e6-888f-add43e477fc7" Jun 07 09:19:49.406596 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.406560 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-gzjfp"] Jun 07 09:19:49.409161 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.409135 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:49.412050 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.412024 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 07 09:19:49.414679 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.414653 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd5ml\" (UniqueName: \"kubernetes.io/projected/909b721e-40bc-46e6-888f-add43e477fc7-kube-api-access-dd5ml\") pod \"authorino-5d4b67c446-hm4gj\" (UID: \"909b721e-40bc-46e6-888f-add43e477fc7\") " pod="kuadrant-system/authorino-5d4b67c446-hm4gj" Jun 07 09:19:49.417599 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.417577 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-gzjfp"] Jun 07 09:19:49.423292 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.423272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd5ml\" (UniqueName: \"kubernetes.io/projected/909b721e-40bc-46e6-888f-add43e477fc7-kube-api-access-dd5ml\") pod \"authorino-5d4b67c446-hm4gj\" (UID: \"909b721e-40bc-46e6-888f-add43e477fc7\") " pod="kuadrant-system/authorino-5d4b67c446-hm4gj" Jun 07 09:19:49.451693 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.451613 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-gzjfp"] Jun 07 09:19:49.451910 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:19:49.451891 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7hlr9 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-gzjfp" podUID="2a659acb-ab18-41e5-8d24-7cc8282df599" Jun 07 09:19:49.479507 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.479474 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7754bf6954-vwq2m"] Jun 07 09:19:49.482049 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.482028 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:19:49.491782 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.491750 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7754bf6954-vwq2m"] Jun 07 09:19:49.516292 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.516258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hlr9\" (UniqueName: \"kubernetes.io/projected/2a659acb-ab18-41e5-8d24-7cc8282df599-kube-api-access-7hlr9\") pod \"authorino-76877994-gzjfp\" (UID: \"2a659acb-ab18-41e5-8d24-7cc8282df599\") " pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:49.516452 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.516310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a659acb-ab18-41e5-8d24-7cc8282df599-tls-cert\") pod \"authorino-76877994-gzjfp\" (UID: \"2a659acb-ab18-41e5-8d24-7cc8282df599\") " pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:49.617575 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.617542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njh7x\" (UniqueName: \"kubernetes.io/projected/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-kube-api-access-njh7x\") pod \"authorino-7754bf6954-vwq2m\" (UID: \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\") " pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:19:49.618041 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.617597 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hlr9\" (UniqueName: \"kubernetes.io/projected/2a659acb-ab18-41e5-8d24-7cc8282df599-kube-api-access-7hlr9\") pod \"authorino-76877994-gzjfp\" (UID: \"2a659acb-ab18-41e5-8d24-7cc8282df599\") " pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:49.618041 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.617645 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-tls-cert\") pod \"authorino-7754bf6954-vwq2m\" (UID: \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\") " pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:19:49.618041 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.617704 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a659acb-ab18-41e5-8d24-7cc8282df599-tls-cert\") pod \"authorino-76877994-gzjfp\" (UID: \"2a659acb-ab18-41e5-8d24-7cc8282df599\") " pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:49.620180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.620157 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a659acb-ab18-41e5-8d24-7cc8282df599-tls-cert\") pod \"authorino-76877994-gzjfp\" (UID: \"2a659acb-ab18-41e5-8d24-7cc8282df599\") " pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:49.628326 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.628306 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hlr9\" (UniqueName: \"kubernetes.io/projected/2a659acb-ab18-41e5-8d24-7cc8282df599-kube-api-access-7hlr9\") pod \"authorino-76877994-gzjfp\" (UID: \"2a659acb-ab18-41e5-8d24-7cc8282df599\") " pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:49.719110 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.719029 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njh7x\" (UniqueName: \"kubernetes.io/projected/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-kube-api-access-njh7x\") pod \"authorino-7754bf6954-vwq2m\" (UID: \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\") " pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:19:49.719110 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.719086 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-tls-cert\") pod \"authorino-7754bf6954-vwq2m\" (UID: \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\") " pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:19:49.721432 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.721413 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-tls-cert\") pod \"authorino-7754bf6954-vwq2m\" (UID: \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\") " pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:19:49.726533 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.726512 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njh7x\" (UniqueName: \"kubernetes.io/projected/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-kube-api-access-njh7x\") pod \"authorino-7754bf6954-vwq2m\" (UID: \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\") " pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:19:49.792975 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.792900 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:19:49.919461 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:49.919434 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7754bf6954-vwq2m"] Jun 07 09:19:49.921417 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:19:49.921389 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba3a29d3_2f3c_4ac3_9cf3_46f2935d005a.slice/crio-c3feebcb71b3b614fa556b75df98a8f5efc5af190acc1abc487dbc1628a4d920 WatchSource:0}: Error finding container c3feebcb71b3b614fa556b75df98a8f5efc5af190acc1abc487dbc1628a4d920: Status 404 returned error can't find the container with id c3feebcb71b3b614fa556b75df98a8f5efc5af190acc1abc487dbc1628a4d920 Jun 07 09:19:50.298033 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.298011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:50.298289 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.298012 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hm4gj" Jun 07 09:19:50.299517 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.299492 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7754bf6954-vwq2m" event={"ID":"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a","Type":"ContainerStarted","Data":"c3feebcb71b3b614fa556b75df98a8f5efc5af190acc1abc487dbc1628a4d920"} Jun 07 09:19:50.305859 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.305837 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:50.307501 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.307487 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hm4gj" Jun 07 09:19:50.425236 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.425210 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hlr9\" (UniqueName: \"kubernetes.io/projected/2a659acb-ab18-41e5-8d24-7cc8282df599-kube-api-access-7hlr9\") pod \"2a659acb-ab18-41e5-8d24-7cc8282df599\" (UID: \"2a659acb-ab18-41e5-8d24-7cc8282df599\") " Jun 07 09:19:50.425678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.425273 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd5ml\" (UniqueName: \"kubernetes.io/projected/909b721e-40bc-46e6-888f-add43e477fc7-kube-api-access-dd5ml\") pod \"909b721e-40bc-46e6-888f-add43e477fc7\" (UID: \"909b721e-40bc-46e6-888f-add43e477fc7\") " Jun 07 09:19:50.425678 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.425297 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a659acb-ab18-41e5-8d24-7cc8282df599-tls-cert\") pod \"2a659acb-ab18-41e5-8d24-7cc8282df599\" (UID: \"2a659acb-ab18-41e5-8d24-7cc8282df599\") " Jun 07 09:19:50.427613 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.427584 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a659acb-ab18-41e5-8d24-7cc8282df599-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2a659acb-ab18-41e5-8d24-7cc8282df599" (UID: "2a659acb-ab18-41e5-8d24-7cc8282df599"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:19:50.427839 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.427812 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/909b721e-40bc-46e6-888f-add43e477fc7-kube-api-access-dd5ml" (OuterVolumeSpecName: "kube-api-access-dd5ml") pod "909b721e-40bc-46e6-888f-add43e477fc7" (UID: "909b721e-40bc-46e6-888f-add43e477fc7"). InnerVolumeSpecName "kube-api-access-dd5ml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:19:50.427955 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.427936 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a659acb-ab18-41e5-8d24-7cc8282df599-kube-api-access-7hlr9" (OuterVolumeSpecName: "kube-api-access-7hlr9") pod "2a659acb-ab18-41e5-8d24-7cc8282df599" (UID: "2a659acb-ab18-41e5-8d24-7cc8282df599"). InnerVolumeSpecName "kube-api-access-7hlr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:19:50.526321 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.526282 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd5ml\" (UniqueName: \"kubernetes.io/projected/909b721e-40bc-46e6-888f-add43e477fc7-kube-api-access-dd5ml\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:19:50.526321 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.526316 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2a659acb-ab18-41e5-8d24-7cc8282df599-tls-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:19:50.526321 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:50.526327 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hlr9\" (UniqueName: \"kubernetes.io/projected/2a659acb-ab18-41e5-8d24-7cc8282df599-kube-api-access-7hlr9\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:19:51.304242 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.304207 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7754bf6954-vwq2m" event={"ID":"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a","Type":"ContainerStarted","Data":"1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0"} Jun 07 09:19:51.304423 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.304301 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-gzjfp" Jun 07 09:19:51.304423 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.304413 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-hm4gj" Jun 07 09:19:51.320495 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.320446 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7754bf6954-vwq2m" podStartSLOduration=1.9021626839999999 podStartE2EDuration="2.320429913s" totalImagesPullingTime="418.267229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:19:51.319398929 +0000 UTC m=+703.464459849" watchObservedRunningTime="2026-06-07 09:19:51.320429913 +0000 UTC m=+703.465490848" Jun 07 09:19:51.341242 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.341209 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-gzjfp"] Jun 07 09:19:51.349013 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.348989 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-gzjfp"] Jun 07 09:19:51.351590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.351561 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k7zt6"] Jun 07 09:19:51.351786 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.351759 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-k7zt6" podUID="9dfb8592-ed72-480a-91bb-898703ec31e9" containerName="authorino" containerID="cri-o://ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d" gracePeriod=30 Jun 07 09:19:51.368676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.368650 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hm4gj"] Jun 07 09:19:51.375450 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.375428 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-hm4gj"] Jun 07 09:19:51.603205 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.603180 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k7zt6" Jun 07 09:19:51.634785 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.634753 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6lkh\" (UniqueName: \"kubernetes.io/projected/9dfb8592-ed72-480a-91bb-898703ec31e9-kube-api-access-x6lkh\") pod \"9dfb8592-ed72-480a-91bb-898703ec31e9\" (UID: \"9dfb8592-ed72-480a-91bb-898703ec31e9\") " Jun 07 09:19:51.636749 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.636718 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dfb8592-ed72-480a-91bb-898703ec31e9-kube-api-access-x6lkh" (OuterVolumeSpecName: "kube-api-access-x6lkh") pod "9dfb8592-ed72-480a-91bb-898703ec31e9" (UID: "9dfb8592-ed72-480a-91bb-898703ec31e9"). InnerVolumeSpecName "kube-api-access-x6lkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:19:51.735819 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:51.735780 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6lkh\" (UniqueName: \"kubernetes.io/projected/9dfb8592-ed72-480a-91bb-898703ec31e9-kube-api-access-x6lkh\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:19:52.299541 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.299512 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a659acb-ab18-41e5-8d24-7cc8282df599" path="/var/lib/kubelet/pods/2a659acb-ab18-41e5-8d24-7cc8282df599/volumes" Jun 07 09:19:52.299749 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.299737 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="909b721e-40bc-46e6-888f-add43e477fc7" path="/var/lib/kubelet/pods/909b721e-40bc-46e6-888f-add43e477fc7/volumes" Jun 07 09:19:52.310028 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.309994 2578 generic.go:356] "Generic (PLEG): container finished" podID="9dfb8592-ed72-480a-91bb-898703ec31e9" containerID="ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d" exitCode=0 Jun 07 09:19:52.310150 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.310043 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k7zt6" Jun 07 09:19:52.310150 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.310085 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k7zt6" event={"ID":"9dfb8592-ed72-480a-91bb-898703ec31e9","Type":"ContainerDied","Data":"ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d"} Jun 07 09:19:52.310150 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.310131 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k7zt6" event={"ID":"9dfb8592-ed72-480a-91bb-898703ec31e9","Type":"ContainerDied","Data":"46c9d71a6b241c0e7335c86accf0896a00e498bbec20f7da4756b2cbd4942294"} Jun 07 09:19:52.310150 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.310150 2578 scope.go:117] "RemoveContainer" containerID="ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d" Jun 07 09:19:52.318911 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.318899 2578 scope.go:117] "RemoveContainer" containerID="ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d" Jun 07 09:19:52.319191 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:19:52.319172 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d\": container with ID starting with ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d not found: ID does not exist" containerID="ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d" Jun 07 09:19:52.319239 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.319202 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d"} err="failed to get container status \"ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d\": rpc error: code = NotFound desc = could not find container \"ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d\": container with ID starting with ecfc9e3f165b95b66d54520a033d98bd343f8e122ba955ae8307f7a1d00f806d not found: ID does not exist" Jun 07 09:19:52.326701 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.326678 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k7zt6"] Jun 07 09:19:52.334053 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:52.328882 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k7zt6"] Jun 07 09:19:54.302515 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.302427 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dfb8592-ed72-480a-91bb-898703ec31e9" path="/var/lib/kubelet/pods/9dfb8592-ed72-480a-91bb-898703ec31e9/volumes" Jun 07 09:19:54.851971 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.851940 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-89fffb9f5-wctpl"] Jun 07 09:19:54.852337 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.852326 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dfb8592-ed72-480a-91bb-898703ec31e9" containerName="authorino" Jun 07 09:19:54.852380 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.852339 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dfb8592-ed72-480a-91bb-898703ec31e9" containerName="authorino" Jun 07 09:19:54.852462 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.852407 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dfb8592-ed72-480a-91bb-898703ec31e9" containerName="authorino" Jun 07 09:19:54.855373 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.855351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:54.858268 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.858247 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 07 09:19:54.858414 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.858399 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 07 09:19:54.858493 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.858479 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6vrm4\"" Jun 07 09:19:54.859979 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.859956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vss8\" (UniqueName: \"kubernetes.io/projected/8744f037-06c4-479b-88c9-43dc021dfa27-kube-api-access-8vss8\") pod \"maas-controller-89fffb9f5-wctpl\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:54.860097 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.860080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert\") pod \"maas-controller-89fffb9f5-wctpl\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:54.870008 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.869984 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-89fffb9f5-wctpl"] Jun 07 09:19:54.960934 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.960892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert\") pod \"maas-controller-89fffb9f5-wctpl\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:54.961130 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.960979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vss8\" (UniqueName: \"kubernetes.io/projected/8744f037-06c4-479b-88c9-43dc021dfa27-kube-api-access-8vss8\") pod \"maas-controller-89fffb9f5-wctpl\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:54.961130 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:19:54.961053 2578 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 07 09:19:54.961130 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:19:54.961128 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert podName:8744f037-06c4-479b-88c9-43dc021dfa27 nodeName:}" failed. No retries permitted until 2026-06-07 09:19:55.461109209 +0000 UTC m=+707.606170108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert") pod "maas-controller-89fffb9f5-wctpl" (UID: "8744f037-06c4-479b-88c9-43dc021dfa27") : secret "maas-controller-webhook-cert" not found Jun 07 09:19:54.971107 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:54.971077 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vss8\" (UniqueName: \"kubernetes.io/projected/8744f037-06c4-479b-88c9-43dc021dfa27-kube-api-access-8vss8\") pod \"maas-controller-89fffb9f5-wctpl\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:55.008213 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.008179 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-545c76b485-qcxrg"] Jun 07 09:19:55.010689 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.010671 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:55.019029 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.019004 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-545c76b485-qcxrg"] Jun 07 09:19:55.062198 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.062158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4stq\" (UniqueName: \"kubernetes.io/projected/49264035-b4d1-4fd7-a12c-14ae34136a03-kube-api-access-j4stq\") pod \"maas-controller-545c76b485-qcxrg\" (UID: \"49264035-b4d1-4fd7-a12c-14ae34136a03\") " pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:55.062198 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.062200 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49264035-b4d1-4fd7-a12c-14ae34136a03-webhook-cert\") pod \"maas-controller-545c76b485-qcxrg\" (UID: \"49264035-b4d1-4fd7-a12c-14ae34136a03\") " pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:55.163245 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.163155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4stq\" (UniqueName: \"kubernetes.io/projected/49264035-b4d1-4fd7-a12c-14ae34136a03-kube-api-access-j4stq\") pod \"maas-controller-545c76b485-qcxrg\" (UID: \"49264035-b4d1-4fd7-a12c-14ae34136a03\") " pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:55.163245 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.163198 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49264035-b4d1-4fd7-a12c-14ae34136a03-webhook-cert\") pod \"maas-controller-545c76b485-qcxrg\" (UID: \"49264035-b4d1-4fd7-a12c-14ae34136a03\") " pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:55.165824 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.165792 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49264035-b4d1-4fd7-a12c-14ae34136a03-webhook-cert\") pod \"maas-controller-545c76b485-qcxrg\" (UID: \"49264035-b4d1-4fd7-a12c-14ae34136a03\") " pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:55.170851 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.170824 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4stq\" (UniqueName: \"kubernetes.io/projected/49264035-b4d1-4fd7-a12c-14ae34136a03-kube-api-access-j4stq\") pod \"maas-controller-545c76b485-qcxrg\" (UID: \"49264035-b4d1-4fd7-a12c-14ae34136a03\") " pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:55.322168 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.322129 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:55.454533 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.454512 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-545c76b485-qcxrg"] Jun 07 09:19:55.457539 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:19:55.457514 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49264035_b4d1_4fd7_a12c_14ae34136a03.slice/crio-77d5977ad3537b5fa5a1f31be5cfa87f978560b118fa85f70ef9ff9387170533 WatchSource:0}: Error finding container 77d5977ad3537b5fa5a1f31be5cfa87f978560b118fa85f70ef9ff9387170533: Status 404 returned error can't find the container with id 77d5977ad3537b5fa5a1f31be5cfa87f978560b118fa85f70ef9ff9387170533 Jun 07 09:19:55.459057 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.459038 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 07 09:19:55.465381 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.465358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert\") pod \"maas-controller-89fffb9f5-wctpl\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:55.467573 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.467552 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert\") pod \"maas-controller-89fffb9f5-wctpl\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:55.767413 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.767370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:55.892902 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:55.892876 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-89fffb9f5-wctpl"] Jun 07 09:19:55.894765 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:19:55.894729 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8744f037_06c4_479b_88c9_43dc021dfa27.slice/crio-31942fb2e8bceb557c4fe41488424516c5d3b43e55af74c5cea9700c1a0b1b08 WatchSource:0}: Error finding container 31942fb2e8bceb557c4fe41488424516c5d3b43e55af74c5cea9700c1a0b1b08: Status 404 returned error can't find the container with id 31942fb2e8bceb557c4fe41488424516c5d3b43e55af74c5cea9700c1a0b1b08 Jun 07 09:19:56.335685 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:56.335584 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-89fffb9f5-wctpl" event={"ID":"8744f037-06c4-479b-88c9-43dc021dfa27","Type":"ContainerStarted","Data":"31942fb2e8bceb557c4fe41488424516c5d3b43e55af74c5cea9700c1a0b1b08"} Jun 07 09:19:56.337516 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:56.337465 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-545c76b485-qcxrg" event={"ID":"49264035-b4d1-4fd7-a12c-14ae34136a03","Type":"ContainerStarted","Data":"77d5977ad3537b5fa5a1f31be5cfa87f978560b118fa85f70ef9ff9387170533"} Jun 07 09:19:58.348838 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:58.348791 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-545c76b485-qcxrg" event={"ID":"49264035-b4d1-4fd7-a12c-14ae34136a03","Type":"ContainerStarted","Data":"62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45"} Jun 07 09:19:58.349428 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:58.348979 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:19:58.350608 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:58.350581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-89fffb9f5-wctpl" event={"ID":"8744f037-06c4-479b-88c9-43dc021dfa27","Type":"ContainerStarted","Data":"4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b"} Jun 07 09:19:58.350758 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:58.350724 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:19:58.388979 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:58.388869 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-545c76b485-qcxrg" podStartSLOduration=1.7186143839999999 podStartE2EDuration="4.388852764s" totalImagesPullingTime="2.67023838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:19:58.387444439 +0000 UTC m=+710.532505359" watchObservedRunningTime="2026-06-07 09:19:58.388852764 +0000 UTC m=+710.533913684" Jun 07 09:19:58.417734 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:19:58.417690 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-89fffb9f5-wctpl" podStartSLOduration=2.17713807 podStartE2EDuration="4.417675654s" totalImagesPullingTime="2.240537584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:19:58.416223676 +0000 UTC m=+710.561284599" watchObservedRunningTime="2026-06-07 09:19:58.417675654 +0000 UTC m=+710.562736573" Jun 07 09:20:09.364970 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.363103 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:20:09.364970 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.364443 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:20:09.428939 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.428900 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-89fffb9f5-wctpl"] Jun 07 09:20:09.429147 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.429123 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-89fffb9f5-wctpl" podUID="8744f037-06c4-479b-88c9-43dc021dfa27" containerName="manager" containerID="cri-o://4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b" gracePeriod=30 Jun 07 09:20:09.687672 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.687649 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:20:09.798401 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.798374 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vss8\" (UniqueName: \"kubernetes.io/projected/8744f037-06c4-479b-88c9-43dc021dfa27-kube-api-access-8vss8\") pod \"8744f037-06c4-479b-88c9-43dc021dfa27\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " Jun 07 09:20:09.798559 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.798491 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert\") pod \"8744f037-06c4-479b-88c9-43dc021dfa27\" (UID: \"8744f037-06c4-479b-88c9-43dc021dfa27\") " Jun 07 09:20:09.800379 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.800346 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8744f037-06c4-479b-88c9-43dc021dfa27-kube-api-access-8vss8" (OuterVolumeSpecName: "kube-api-access-8vss8") pod "8744f037-06c4-479b-88c9-43dc021dfa27" (UID: "8744f037-06c4-479b-88c9-43dc021dfa27"). InnerVolumeSpecName "kube-api-access-8vss8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:20:09.800379 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.800369 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8744f037-06c4-479b-88c9-43dc021dfa27" (UID: "8744f037-06c4-479b-88c9-43dc021dfa27"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:20:09.899796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.899766 2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8744f037-06c4-479b-88c9-43dc021dfa27-webhook-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:20:09.899796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:09.899792 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vss8\" (UniqueName: \"kubernetes.io/projected/8744f037-06c4-479b-88c9-43dc021dfa27-kube-api-access-8vss8\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:20:10.409326 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.409294 2578 generic.go:356] "Generic (PLEG): container finished" podID="8744f037-06c4-479b-88c9-43dc021dfa27" containerID="4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b" exitCode=0 Jun 07 09:20:10.409747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.409342 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-89fffb9f5-wctpl" Jun 07 09:20:10.409747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.409360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-89fffb9f5-wctpl" event={"ID":"8744f037-06c4-479b-88c9-43dc021dfa27","Type":"ContainerDied","Data":"4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b"} Jun 07 09:20:10.409747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.409403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-89fffb9f5-wctpl" event={"ID":"8744f037-06c4-479b-88c9-43dc021dfa27","Type":"ContainerDied","Data":"31942fb2e8bceb557c4fe41488424516c5d3b43e55af74c5cea9700c1a0b1b08"} Jun 07 09:20:10.409747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.409423 2578 scope.go:117] "RemoveContainer" containerID="4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b" Jun 07 09:20:10.418216 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.418200 2578 scope.go:117] "RemoveContainer" containerID="4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b" Jun 07 09:20:10.418463 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:20:10.418442 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b\": container with ID starting with 4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b not found: ID does not exist" containerID="4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b" Jun 07 09:20:10.418520 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.418476 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b"} err="failed to get container status \"4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b\": rpc error: code = NotFound desc = could not find container \"4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b\": container with ID starting with 4bec13ea5261133d46794dfd2fe95b5c78445c8bad500f9edbbc791e7779652b not found: ID does not exist" Jun 07 09:20:10.426030 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.426008 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-89fffb9f5-wctpl"] Jun 07 09:20:10.431313 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:10.431293 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-89fffb9f5-wctpl"] Jun 07 09:20:12.299427 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:12.299395 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8744f037-06c4-479b-88c9-43dc021dfa27" path="/var/lib/kubelet/pods/8744f037-06c4-479b-88c9-43dc021dfa27/volumes" Jun 07 09:20:27.781748 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.781709 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-d8cd54b68-wdxsm"] Jun 07 09:20:27.782364 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.782340 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8744f037-06c4-479b-88c9-43dc021dfa27" containerName="manager" Jun 07 09:20:27.782441 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.782367 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8744f037-06c4-479b-88c9-43dc021dfa27" containerName="manager" Jun 07 09:20:27.782494 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.782473 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8744f037-06c4-479b-88c9-43dc021dfa27" containerName="manager" Jun 07 09:20:27.785369 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.785349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:27.788247 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.788224 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 07 09:20:27.788334 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.788224 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-9svnk\"" Jun 07 09:20:27.795568 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.795544 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-d8cd54b68-wdxsm"] Jun 07 09:20:27.958439 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.958404 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43be9139-5ccf-4095-a0da-2b565c0e0f3b-maas-api-tls\") pod \"maas-api-d8cd54b68-wdxsm\" (UID: \"43be9139-5ccf-4095-a0da-2b565c0e0f3b\") " pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:27.958618 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:27.958492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g27rn\" (UniqueName: \"kubernetes.io/projected/43be9139-5ccf-4095-a0da-2b565c0e0f3b-kube-api-access-g27rn\") pod \"maas-api-d8cd54b68-wdxsm\" (UID: \"43be9139-5ccf-4095-a0da-2b565c0e0f3b\") " pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:28.059663 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.059575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g27rn\" (UniqueName: \"kubernetes.io/projected/43be9139-5ccf-4095-a0da-2b565c0e0f3b-kube-api-access-g27rn\") pod \"maas-api-d8cd54b68-wdxsm\" (UID: \"43be9139-5ccf-4095-a0da-2b565c0e0f3b\") " pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:28.059663 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.059641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43be9139-5ccf-4095-a0da-2b565c0e0f3b-maas-api-tls\") pod \"maas-api-d8cd54b68-wdxsm\" (UID: \"43be9139-5ccf-4095-a0da-2b565c0e0f3b\") " pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:28.062217 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.062193 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43be9139-5ccf-4095-a0da-2b565c0e0f3b-maas-api-tls\") pod \"maas-api-d8cd54b68-wdxsm\" (UID: \"43be9139-5ccf-4095-a0da-2b565c0e0f3b\") " pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:28.067127 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.067102 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g27rn\" (UniqueName: \"kubernetes.io/projected/43be9139-5ccf-4095-a0da-2b565c0e0f3b-kube-api-access-g27rn\") pod \"maas-api-d8cd54b68-wdxsm\" (UID: \"43be9139-5ccf-4095-a0da-2b565c0e0f3b\") " pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:28.098048 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.098025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:28.230843 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.230817 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-d8cd54b68-wdxsm"] Jun 07 09:20:28.233244 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:20:28.233216 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43be9139_5ccf_4095_a0da_2b565c0e0f3b.slice/crio-5407265e48c33a99cf173aa87fc9ecfb8fe828b859623840084099ddd95b0f08 WatchSource:0}: Error finding container 5407265e48c33a99cf173aa87fc9ecfb8fe828b859623840084099ddd95b0f08: Status 404 returned error can't find the container with id 5407265e48c33a99cf173aa87fc9ecfb8fe828b859623840084099ddd95b0f08 Jun 07 09:20:28.345748 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.345664 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-wch7k"] Jun 07 09:20:28.350229 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.350208 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" Jun 07 09:20:28.353101 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.353082 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 07 09:20:28.353215 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.353083 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fmnpx\"" Jun 07 09:20:28.359575 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.359548 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-wch7k"] Jun 07 09:20:28.365747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.365726 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-jgjrm"] Jun 07 09:20:28.369045 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.369027 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" Jun 07 09:20:28.381477 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.381452 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-jgjrm"] Jun 07 09:20:28.463062 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.463028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7svvm\" (UniqueName: \"kubernetes.io/projected/36a07749-c4c8-4bf0-baad-9124f99d7a90-kube-api-access-7svvm\") pod \"payload-pre-processing-67d7689b65-wch7k\" (UID: \"36a07749-c4c8-4bf0-baad-9124f99d7a90\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" Jun 07 09:20:28.463236 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.463158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29qgc\" (UniqueName: \"kubernetes.io/projected/7ffe3526-707c-4820-8330-b02bb20821c9-kube-api-access-29qgc\") pod \"payload-processing-7c6c859784-jgjrm\" (UID: \"7ffe3526-707c-4820-8330-b02bb20821c9\") " pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" Jun 07 09:20:28.499782 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.499747 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-d8cd54b68-wdxsm" event={"ID":"43be9139-5ccf-4095-a0da-2b565c0e0f3b","Type":"ContainerStarted","Data":"5407265e48c33a99cf173aa87fc9ecfb8fe828b859623840084099ddd95b0f08"} Jun 07 09:20:28.564215 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.564182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29qgc\" (UniqueName: \"kubernetes.io/projected/7ffe3526-707c-4820-8330-b02bb20821c9-kube-api-access-29qgc\") pod \"payload-processing-7c6c859784-jgjrm\" (UID: \"7ffe3526-707c-4820-8330-b02bb20821c9\") " pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" Jun 07 09:20:28.564377 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.564246 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7svvm\" (UniqueName: \"kubernetes.io/projected/36a07749-c4c8-4bf0-baad-9124f99d7a90-kube-api-access-7svvm\") pod \"payload-pre-processing-67d7689b65-wch7k\" (UID: \"36a07749-c4c8-4bf0-baad-9124f99d7a90\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" Jun 07 09:20:28.572257 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.572232 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29qgc\" (UniqueName: \"kubernetes.io/projected/7ffe3526-707c-4820-8330-b02bb20821c9-kube-api-access-29qgc\") pod \"payload-processing-7c6c859784-jgjrm\" (UID: \"7ffe3526-707c-4820-8330-b02bb20821c9\") " pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" Jun 07 09:20:28.572367 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.572303 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7svvm\" (UniqueName: \"kubernetes.io/projected/36a07749-c4c8-4bf0-baad-9124f99d7a90-kube-api-access-7svvm\") pod \"payload-pre-processing-67d7689b65-wch7k\" (UID: \"36a07749-c4c8-4bf0-baad-9124f99d7a90\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" Jun 07 09:20:28.661748 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.661675 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" Jun 07 09:20:28.680441 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.680416 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" Jun 07 09:20:28.803214 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.803181 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-wch7k"] Jun 07 09:20:28.809009 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:20:28.808977 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a07749_c4c8_4bf0_baad_9124f99d7a90.slice/crio-93a7b22e92e3942b535f78f2a795cf7bee8df09075e503fdd853272e60fe80e5 WatchSource:0}: Error finding container 93a7b22e92e3942b535f78f2a795cf7bee8df09075e503fdd853272e60fe80e5: Status 404 returned error can't find the container with id 93a7b22e92e3942b535f78f2a795cf7bee8df09075e503fdd853272e60fe80e5 Jun 07 09:20:28.830016 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:28.829993 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-jgjrm"] Jun 07 09:20:28.832599 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:20:28.832577 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ffe3526_707c_4820_8330_b02bb20821c9.slice/crio-252c09f9f74690b50728a39e7fd34316de6c730d495bcb47478227056e0a05c6 WatchSource:0}: Error finding container 252c09f9f74690b50728a39e7fd34316de6c730d495bcb47478227056e0a05c6: Status 404 returned error can't find the container with id 252c09f9f74690b50728a39e7fd34316de6c730d495bcb47478227056e0a05c6 Jun 07 09:20:29.509994 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:29.509953 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" event={"ID":"36a07749-c4c8-4bf0-baad-9124f99d7a90","Type":"ContainerStarted","Data":"93a7b22e92e3942b535f78f2a795cf7bee8df09075e503fdd853272e60fe80e5"} Jun 07 09:20:29.512192 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:29.512162 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" event={"ID":"7ffe3526-707c-4820-8330-b02bb20821c9","Type":"ContainerStarted","Data":"252c09f9f74690b50728a39e7fd34316de6c730d495bcb47478227056e0a05c6"} Jun 07 09:20:30.518641 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.518596 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" event={"ID":"7ffe3526-707c-4820-8330-b02bb20821c9","Type":"ContainerStarted","Data":"e2c051fcbaefe06fd7fad6c93c02102a458d1f4455b218fbb14fee78c72698d5"} Jun 07 09:20:30.519090 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.518667 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" Jun 07 09:20:30.520059 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.520030 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-d8cd54b68-wdxsm" event={"ID":"43be9139-5ccf-4095-a0da-2b565c0e0f3b","Type":"ContainerStarted","Data":"21aa7b5df4416eaab5ae0fd291893f89753fc3e2546c5ddaec9ce3ca2f4968ca"} Jun 07 09:20:30.520183 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.520090 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:30.521479 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.521459 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" event={"ID":"36a07749-c4c8-4bf0-baad-9124f99d7a90","Type":"ContainerStarted","Data":"4b64e85a538bba2c5c8d098f0b3923c70384533fa82b6fb40679fedd63c910e3"} Jun 07 09:20:30.521585 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.521564 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" Jun 07 09:20:30.535776 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.535733 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" podStartSLOduration=1.134696275 podStartE2EDuration="2.535715566s" totalImagesPullingTime="1.401019291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:20:30.53449711 +0000 UTC m=+742.679558031" watchObservedRunningTime="2026-06-07 09:20:30.535715566 +0000 UTC m=+742.680776479" Jun 07 09:20:30.552649 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.552602 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" podStartSLOduration=1.13056011 podStartE2EDuration="2.552583359s" totalImagesPullingTime="1.422023249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:20:30.550570961 +0000 UTC m=+742.695631881" watchObservedRunningTime="2026-06-07 09:20:30.552583359 +0000 UTC m=+742.697644287" Jun 07 09:20:30.569481 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:30.569438 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-d8cd54b68-wdxsm" podStartSLOduration=1.627442453 podStartE2EDuration="3.569425322s" totalImagesPullingTime="1.941982869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:20:30.567455241 +0000 UTC m=+742.712516150" watchObservedRunningTime="2026-06-07 09:20:30.569425322 +0000 UTC m=+742.714486240" Jun 07 09:20:36.533587 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:36.533563 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-d8cd54b68-wdxsm" Jun 07 09:20:38.343169 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:38.343135 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:20:38.343560 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:38.343323 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" containerID="cri-o://5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219" gracePeriod=30 Jun 07 09:20:40.391905 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.391877 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.464913 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.464886 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6g8h\" (UniqueName: \"kubernetes.io/projected/6c6eab1b-1adb-439e-bb19-85db8b5432e6-kube-api-access-n6g8h\") pod \"6c6eab1b-1adb-439e-bb19-85db8b5432e6\" (UID: \"6c6eab1b-1adb-439e-bb19-85db8b5432e6\") " Jun 07 09:20:40.466894 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.466871 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c6eab1b-1adb-439e-bb19-85db8b5432e6-kube-api-access-n6g8h" (OuterVolumeSpecName: "kube-api-access-n6g8h") pod "6c6eab1b-1adb-439e-bb19-85db8b5432e6" (UID: "6c6eab1b-1adb-439e-bb19-85db8b5432e6"). InnerVolumeSpecName "kube-api-access-n6g8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:20:40.566339 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.566269 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6g8h\" (UniqueName: \"kubernetes.io/projected/6c6eab1b-1adb-439e-bb19-85db8b5432e6-kube-api-access-n6g8h\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:20:40.580655 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.580629 2578 generic.go:356] "Generic (PLEG): container finished" podID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerID="5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219" exitCode=143 Jun 07 09:20:40.580796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.580670 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.580796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.580715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"6c6eab1b-1adb-439e-bb19-85db8b5432e6","Type":"ContainerDied","Data":"5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219"} Jun 07 09:20:40.580796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.580764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"6c6eab1b-1adb-439e-bb19-85db8b5432e6","Type":"ContainerDied","Data":"36a5e9d4398574cd1c5eb3b18f069e2dff42321c44e97a3ffd3a90b5f783fa9d"} Jun 07 09:20:40.580796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.580791 2578 scope.go:117] "RemoveContainer" containerID="5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219" Jun 07 09:20:40.590397 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.590378 2578 scope.go:117] "RemoveContainer" containerID="5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219" Jun 07 09:20:40.590629 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:20:40.590609 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219\": container with ID starting with 5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219 not found: ID does not exist" containerID="5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219" Jun 07 09:20:40.590681 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.590637 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219"} err="failed to get container status \"5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219\": rpc error: code = NotFound desc = could not find container \"5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219\": container with ID starting with 5e27c51308141fbf37ef0c7970513dcf0de0e0f922f0e8d7382af703e5828219 not found: ID does not exist" Jun 07 09:20:40.602346 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.602319 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:20:40.606017 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.605991 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:20:40.625810 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.625785 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:20:40.626187 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.626174 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" Jun 07 09:20:40.626239 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.626189 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" Jun 07 09:20:40.626284 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.626275 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" containerName="keycloak" Jun 07 09:20:40.630347 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.630329 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.633169 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.633150 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 07 09:20:40.633316 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.633299 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 07 09:20:40.633386 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.633323 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-vcxn7\"" Jun 07 09:20:40.633456 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.633383 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 07 09:20:40.633456 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.633419 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 07 09:20:40.639272 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.639245 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:20:40.666709 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.666685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smzb5\" (UniqueName: \"kubernetes.io/projected/d0ad6ab8-9f34-4f0f-94f3-adb97db40359-kube-api-access-smzb5\") pod \"maas-keycloak-0\" (UID: \"d0ad6ab8-9f34-4f0f-94f3-adb97db40359\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.666801 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.666724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/d0ad6ab8-9f34-4f0f-94f3-adb97db40359-test-realms\") pod \"maas-keycloak-0\" (UID: \"d0ad6ab8-9f34-4f0f-94f3-adb97db40359\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.767343 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.767313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smzb5\" (UniqueName: \"kubernetes.io/projected/d0ad6ab8-9f34-4f0f-94f3-adb97db40359-kube-api-access-smzb5\") pod \"maas-keycloak-0\" (UID: \"d0ad6ab8-9f34-4f0f-94f3-adb97db40359\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.767496 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.767352 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/d0ad6ab8-9f34-4f0f-94f3-adb97db40359-test-realms\") pod \"maas-keycloak-0\" (UID: \"d0ad6ab8-9f34-4f0f-94f3-adb97db40359\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.768037 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.768019 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/d0ad6ab8-9f34-4f0f-94f3-adb97db40359-test-realms\") pod \"maas-keycloak-0\" (UID: \"d0ad6ab8-9f34-4f0f-94f3-adb97db40359\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.775700 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.775677 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smzb5\" (UniqueName: \"kubernetes.io/projected/d0ad6ab8-9f34-4f0f-94f3-adb97db40359-kube-api-access-smzb5\") pod \"maas-keycloak-0\" (UID: \"d0ad6ab8-9f34-4f0f-94f3-adb97db40359\") " pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:40.940315 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:40.940222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:41.073892 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:41.073869 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 07 09:20:41.074669 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:20:41.074643 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ad6ab8_9f34_4f0f_94f3_adb97db40359.slice/crio-08466c834c5023788d48c4d7a9ee28d71b0b4492637443a86658653c40b420b4 WatchSource:0}: Error finding container 08466c834c5023788d48c4d7a9ee28d71b0b4492637443a86658653c40b420b4: Status 404 returned error can't find the container with id 08466c834c5023788d48c4d7a9ee28d71b0b4492637443a86658653c40b420b4 Jun 07 09:20:41.530326 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:41.530301 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-jgjrm" Jun 07 09:20:41.530698 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:41.530656 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-wch7k" Jun 07 09:20:41.589496 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:41.589455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d0ad6ab8-9f34-4f0f-94f3-adb97db40359","Type":"ContainerStarted","Data":"e2822f226c47d3f44686f5df3077673b48c00ed94d7f49fdc236aed544ec94ca"} Jun 07 09:20:41.589676 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:41.589546 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d0ad6ab8-9f34-4f0f-94f3-adb97db40359","Type":"ContainerStarted","Data":"08466c834c5023788d48c4d7a9ee28d71b0b4492637443a86658653c40b420b4"} Jun 07 09:20:41.609283 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:41.609225 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.251833784 podStartE2EDuration="1.609203594s" totalImagesPullingTime="357.36981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:20:41.606038226 +0000 UTC m=+753.751099145" watchObservedRunningTime="2026-06-07 09:20:41.609203594 +0000 UTC m=+753.754264514" Jun 07 09:20:41.940615 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:41.940573 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:41.942290 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:41.942242 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:42.301110 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:42.301078 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c6eab1b-1adb-439e-bb19-85db8b5432e6" path="/var/lib/kubelet/pods/6c6eab1b-1adb-439e-bb19-85db8b5432e6/volumes" Jun 07 09:20:42.941417 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:42.941360 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:43.941794 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:43.941731 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:44.941802 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:44.941745 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:45.941325 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:45.941280 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:46.941420 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:46.941372 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:47.940744 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:47.940693 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:48.940758 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:48.940711 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:49.941173 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:49.941120 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:50.940849 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:50.940811 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:50.941217 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:50.941181 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:51.940762 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:51.940712 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:52.941252 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:52.941209 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:53.940886 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:53.940839 2578 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 07 09:20:55.063180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:55.063128 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 07 09:20:55.080557 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:20:55.080505 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="d0ad6ab8-9f34-4f0f-94f3-adb97db40359" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:21:05.069814 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:05.069781 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 07 09:21:15.744485 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.744452 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-566c97dd5b-ktfvc"] Jun 07 09:21:15.750001 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.749984 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.753218 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.753185 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 07 09:21:15.754683 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.754659 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-566c97dd5b-ktfvc"] Jun 07 09:21:15.812272 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.812234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-tls-cert\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.812446 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.812292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-oidc-ca\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.812446 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.812421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j48v6\" (UniqueName: \"kubernetes.io/projected/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-kube-api-access-j48v6\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.913202 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.913166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-oidc-ca\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.913398 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.913262 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j48v6\" (UniqueName: \"kubernetes.io/projected/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-kube-api-access-j48v6\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.913398 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.913316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-tls-cert\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.913840 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.913812 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-oidc-ca\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.915758 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.915741 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-tls-cert\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:15.920669 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:15.920647 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j48v6\" (UniqueName: \"kubernetes.io/projected/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-kube-api-access-j48v6\") pod \"authorino-566c97dd5b-ktfvc\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:16.060531 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:16.060451 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:21:16.189459 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:16.189430 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-566c97dd5b-ktfvc"] Jun 07 09:21:16.190787 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:21:16.190759 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1898e2a_8c2c_4e41_b2c9_b1ee470b4e68.slice/crio-3d4c503686de926cf7a05c222c46f80eb10b4008370c2f9232daa4cbda962598 WatchSource:0}: Error finding container 3d4c503686de926cf7a05c222c46f80eb10b4008370c2f9232daa4cbda962598: Status 404 returned error can't find the container with id 3d4c503686de926cf7a05c222c46f80eb10b4008370c2f9232daa4cbda962598 Jun 07 09:21:16.804110 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:16.804008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" event={"ID":"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68","Type":"ContainerStarted","Data":"2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2"} Jun 07 09:21:16.804110 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:16.804066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" event={"ID":"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68","Type":"ContainerStarted","Data":"3d4c503686de926cf7a05c222c46f80eb10b4008370c2f9232daa4cbda962598"} Jun 07 09:21:16.822834 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:16.822778 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" podStartSLOduration=1.467195012 podStartE2EDuration="1.822761738s" totalImagesPullingTime="355.566726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:21:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:21:16.819828071 +0000 UTC m=+788.964888992" watchObservedRunningTime="2026-06-07 09:21:16.822761738 +0000 UTC m=+788.967822657" Jun 07 09:21:16.851519 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:16.850226 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7754bf6954-vwq2m"] Jun 07 09:21:16.851519 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:16.850439 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7754bf6954-vwq2m" podUID="ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a" containerName="authorino" containerID="cri-o://1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0" gracePeriod=30 Jun 07 09:21:17.121559 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.121533 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:21:17.225971 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.225918 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-tls-cert\") pod \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\" (UID: \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\") " Jun 07 09:21:17.226156 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.226054 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njh7x\" (UniqueName: \"kubernetes.io/projected/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-kube-api-access-njh7x\") pod \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\" (UID: \"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a\") " Jun 07 09:21:17.228018 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.227986 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-kube-api-access-njh7x" (OuterVolumeSpecName: "kube-api-access-njh7x") pod "ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a" (UID: "ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a"). InnerVolumeSpecName "kube-api-access-njh7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:21:17.236899 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.236871 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a" (UID: "ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:21:17.327302 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.327071 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njh7x\" (UniqueName: \"kubernetes.io/projected/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-kube-api-access-njh7x\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:21:17.327302 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.327103 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a-tls-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:21:17.810255 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.810224 2578 generic.go:356] "Generic (PLEG): container finished" podID="ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a" containerID="1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0" exitCode=0 Jun 07 09:21:17.810662 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.810265 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7754bf6954-vwq2m" Jun 07 09:21:17.810662 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.810302 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7754bf6954-vwq2m" event={"ID":"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a","Type":"ContainerDied","Data":"1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0"} Jun 07 09:21:17.810662 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.810332 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7754bf6954-vwq2m" event={"ID":"ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a","Type":"ContainerDied","Data":"c3feebcb71b3b614fa556b75df98a8f5efc5af190acc1abc487dbc1628a4d920"} Jun 07 09:21:17.810662 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.810350 2578 scope.go:117] "RemoveContainer" containerID="1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0" Jun 07 09:21:17.819552 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.819533 2578 scope.go:117] "RemoveContainer" containerID="1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0" Jun 07 09:21:17.819807 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:21:17.819788 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0\": container with ID starting with 1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0 not found: ID does not exist" containerID="1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0" Jun 07 09:21:17.819872 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.819820 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0"} err="failed to get container status \"1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0\": rpc error: code = NotFound desc = could not find container \"1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0\": container with ID starting with 1cdc9471ba9b00efdc6745580c2cfe3758d70289392bcb2b322038bc8f5e75d0 not found: ID does not exist" Jun 07 09:21:17.831225 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.831194 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7754bf6954-vwq2m"] Jun 07 09:21:17.835450 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:17.835428 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7754bf6954-vwq2m"] Jun 07 09:21:18.307443 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:18.307407 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a" path="/var/lib/kubelet/pods/ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a/volumes" Jun 07 09:21:24.172825 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.172630 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm"] Jun 07 09:21:24.173423 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.173400 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a" containerName="authorino" Jun 07 09:21:24.173522 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.173425 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a" containerName="authorino" Jun 07 09:21:24.173581 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.173532 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba3a29d3-2f3c-4ac3-9cf3-46f2935d005a" containerName="authorino" Jun 07 09:21:24.178215 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.178195 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.183750 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.182557 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 07 09:21:24.183750 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.182862 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 07 09:21:24.183750 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.183125 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-htb4v\"" Jun 07 09:21:24.183750 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.183383 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 07 09:21:24.191273 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.191214 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm"] Jun 07 09:21:24.294619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.294590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.294619 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.294620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.294837 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.294647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.294837 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.294673 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.294837 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.294801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbbqq\" (UniqueName: \"kubernetes.io/projected/4e483922-3adb-4390-aed9-ce9d54b774d8-kube-api-access-pbbqq\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.294998 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.294854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e483922-3adb-4390-aed9-ce9d54b774d8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.395349 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.395318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbbqq\" (UniqueName: \"kubernetes.io/projected/4e483922-3adb-4390-aed9-ce9d54b774d8-kube-api-access-pbbqq\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.395526 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.395357 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e483922-3adb-4390-aed9-ce9d54b774d8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.395526 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.395409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.395526 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.395426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.395684 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.395567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.395684 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.395625 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.395912 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.395889 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.396030 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.396010 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.396030 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.396024 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.397791 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.397771 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e483922-3adb-4390-aed9-ce9d54b774d8-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.398097 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.398063 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e483922-3adb-4390-aed9-ce9d54b774d8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.403067 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.403049 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbbqq\" (UniqueName: \"kubernetes.io/projected/4e483922-3adb-4390-aed9-ce9d54b774d8-kube-api-access-pbbqq\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm\" (UID: \"4e483922-3adb-4390-aed9-ce9d54b774d8\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.502353 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.502325 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:24.635169 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.635104 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm"] Jun 07 09:21:24.636966 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:21:24.636938 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e483922_3adb_4390_aed9_ce9d54b774d8.slice/crio-b4f905949169e550f090a18c1ec6499e157efc4e17d83434371912a4ef4dd058 WatchSource:0}: Error finding container b4f905949169e550f090a18c1ec6499e157efc4e17d83434371912a4ef4dd058: Status 404 returned error can't find the container with id b4f905949169e550f090a18c1ec6499e157efc4e17d83434371912a4ef4dd058 Jun 07 09:21:24.847542 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:24.847465 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" event={"ID":"4e483922-3adb-4390-aed9-ce9d54b774d8","Type":"ContainerStarted","Data":"b4f905949169e550f090a18c1ec6499e157efc4e17d83434371912a4ef4dd058"} Jun 07 09:21:28.870687 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:28.870651 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" event={"ID":"4e483922-3adb-4390-aed9-ce9d54b774d8","Type":"ContainerStarted","Data":"6e8469d1623a04f9eaf30fa9f53744a240dbad92792ac575706cc035e29772c0"} Jun 07 09:21:29.877306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:29.877271 2578 generic.go:356] "Generic (PLEG): container finished" podID="4e483922-3adb-4390-aed9-ce9d54b774d8" containerID="6e8469d1623a04f9eaf30fa9f53744a240dbad92792ac575706cc035e29772c0" exitCode=0 Jun 07 09:21:29.877688 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:29.877342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" event={"ID":"4e483922-3adb-4390-aed9-ce9d54b774d8","Type":"ContainerDied","Data":"6e8469d1623a04f9eaf30fa9f53744a240dbad92792ac575706cc035e29772c0"} Jun 07 09:21:31.888364 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:31.888330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" event={"ID":"4e483922-3adb-4390-aed9-ce9d54b774d8","Type":"ContainerStarted","Data":"6a26686b341505bdd71ac59a102510d48a8fd150a0b0be1b91785195f3b526de"} Jun 07 09:21:31.907057 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:31.907009 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" podStartSLOduration=1.964362327 podStartE2EDuration="7.906995213s" totalImagesPullingTime="4.682514896s" totalInitContainerRuntime="1.26011799s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:21:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:21:31.904763735 +0000 UTC m=+804.049824654" watchObservedRunningTime="2026-06-07 09:21:31.906995213 +0000 UTC m=+804.052056132" Jun 07 09:21:34.503311 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:34.503279 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:34.503670 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:34.503328 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:34.516766 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:34.516741 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:34.914549 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:34.914474 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm" Jun 07 09:21:55.187593 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.187560 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht"] Jun 07 09:21:55.204032 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.203983 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht"] Jun 07 09:21:55.204467 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.204245 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.207171 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.207152 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 07 09:21:55.385225 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.385188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.385403 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.385248 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.385403 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.385274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95pql\" (UniqueName: \"kubernetes.io/projected/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-kube-api-access-95pql\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.385403 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.385301 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.385403 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.385322 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.385562 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.385438 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.486495 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.486656 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.486656 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486578 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.486656 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95pql\" (UniqueName: \"kubernetes.io/projected/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-kube-api-access-95pql\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.486656 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486625 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.486656 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.486995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486892 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.486995 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486899 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.487089 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.486999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.488747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.488725 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.488982 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.488963 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.494104 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.494065 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95pql\" (UniqueName: \"kubernetes.io/projected/b9b5a335-503d-4b2e-b6ce-b2561ee0ab74-kube-api-access-95pql\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht\" (UID: \"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.515080 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.515055 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:21:55.648822 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:55.648793 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht"] Jun 07 09:21:55.650377 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:21:55.650354 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b5a335_503d_4b2e_b6ce_b2561ee0ab74.slice/crio-df30223a07d0e1dc3a7480f93f32cc020af3778104584cf9c79e783d2e84cb43 WatchSource:0}: Error finding container df30223a07d0e1dc3a7480f93f32cc020af3778104584cf9c79e783d2e84cb43: Status 404 returned error can't find the container with id df30223a07d0e1dc3a7480f93f32cc020af3778104584cf9c79e783d2e84cb43 Jun 07 09:21:56.008787 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:56.008753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" event={"ID":"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74","Type":"ContainerStarted","Data":"262ed8d5e64248a1481044364e543580c92202f59a51a2893315fcfd07b92156"} Jun 07 09:21:56.008968 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:56.008802 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" event={"ID":"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74","Type":"ContainerStarted","Data":"df30223a07d0e1dc3a7480f93f32cc020af3778104584cf9c79e783d2e84cb43"} Jun 07 09:21:58.020287 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:58.020254 2578 generic.go:356] "Generic (PLEG): container finished" podID="b9b5a335-503d-4b2e-b6ce-b2561ee0ab74" containerID="262ed8d5e64248a1481044364e543580c92202f59a51a2893315fcfd07b92156" exitCode=0 Jun 07 09:21:58.020757 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:58.020331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" event={"ID":"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74","Type":"ContainerDied","Data":"262ed8d5e64248a1481044364e543580c92202f59a51a2893315fcfd07b92156"} Jun 07 09:21:59.027554 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:59.027518 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" event={"ID":"b9b5a335-503d-4b2e-b6ce-b2561ee0ab74","Type":"ContainerStarted","Data":"76161a12db497abf19471de058b5a81223c6c35e8803e5590077753034a74865"} Jun 07 09:21:59.046878 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:21:59.046832 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" podStartSLOduration=2.436254934 podStartE2EDuration="4.046816405s" totalImagesPullingTime="243.329208ms" totalInitContainerRuntime="1.367232263s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:21:59.045649059 +0000 UTC m=+831.190709979" watchObservedRunningTime="2026-06-07 09:21:59.046816405 +0000 UTC m=+831.191877374" Jun 07 09:22:04.878113 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.878078 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6"] Jun 07 09:22:04.882838 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.882821 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:04.885397 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.885374 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 07 09:22:04.890466 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.890430 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6"] Jun 07 09:22:04.968603 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.968567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:04.968776 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.968656 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:04.968776 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.968699 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:04.968776 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.968721 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4090155-f683-424e-aa9b-ca774bccfb4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:04.968884 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.968787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkq79\" (UniqueName: \"kubernetes.io/projected/c4090155-f683-424e-aa9b-ca774bccfb4b-kube-api-access-pkq79\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:04.968884 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:04.968824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.069758 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.069726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.069758 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.069763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.070035 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.069780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4090155-f683-424e-aa9b-ca774bccfb4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.070035 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.069804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkq79\" (UniqueName: \"kubernetes.io/projected/c4090155-f683-424e-aa9b-ca774bccfb4b-kube-api-access-pkq79\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.070035 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.069842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.070035 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.069958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.070282 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.070256 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.070370 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.070345 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.070479 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.070440 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.072209 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.072188 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4090155-f683-424e-aa9b-ca774bccfb4b-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.072422 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.072404 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4090155-f683-424e-aa9b-ca774bccfb4b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.077577 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.077558 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkq79\" (UniqueName: \"kubernetes.io/projected/c4090155-f683-424e-aa9b-ca774bccfb4b-kube-api-access-pkq79\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6\" (UID: \"c4090155-f683-424e-aa9b-ca774bccfb4b\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.195181 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.195097 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:22:05.330385 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.330353 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6"] Jun 07 09:22:05.331515 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:22:05.331485 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4090155_f683_424e_aa9b_ca774bccfb4b.slice/crio-febe04276cf3afbe70b256866cad9a41eaf11322b62aa49961b093a49260c5e3 WatchSource:0}: Error finding container febe04276cf3afbe70b256866cad9a41eaf11322b62aa49961b093a49260c5e3: Status 404 returned error can't find the container with id febe04276cf3afbe70b256866cad9a41eaf11322b62aa49961b093a49260c5e3 Jun 07 09:22:05.516119 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.516090 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:22:05.516306 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.516165 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:22:05.528698 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:05.528674 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:22:06.066870 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:06.066822 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" event={"ID":"c4090155-f683-424e-aa9b-ca774bccfb4b","Type":"ContainerStarted","Data":"acb68833e3b9cf97d62def4dccefa352095093b6d2588e9da1d278cea505cde8"} Jun 07 09:22:06.067430 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:06.066942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" event={"ID":"c4090155-f683-424e-aa9b-ca774bccfb4b","Type":"ContainerStarted","Data":"febe04276cf3afbe70b256866cad9a41eaf11322b62aa49961b093a49260c5e3"} Jun 07 09:22:06.082508 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:06.082450 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht" Jun 07 09:22:58.342840 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:58.342806 2578 generic.go:356] "Generic (PLEG): container finished" podID="c4090155-f683-424e-aa9b-ca774bccfb4b" containerID="acb68833e3b9cf97d62def4dccefa352095093b6d2588e9da1d278cea505cde8" exitCode=0 Jun 07 09:22:58.343212 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:58.342877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" event={"ID":"c4090155-f683-424e-aa9b-ca774bccfb4b","Type":"ContainerDied","Data":"acb68833e3b9cf97d62def4dccefa352095093b6d2588e9da1d278cea505cde8"} Jun 07 09:22:59.349872 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:59.349831 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" event={"ID":"c4090155-f683-424e-aa9b-ca774bccfb4b","Type":"ContainerStarted","Data":"59cc9b408b74ebf51c15845f6c40f202fe710c1659699d1c5b5c32370a053b32"} Jun 07 09:22:59.369871 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:22:59.369818 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" podStartSLOduration=2.194651123 podStartE2EDuration="55.369790534s" totalImagesPullingTime="237.516536ms" totalInitContainerRuntime="52.937622875s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:22:59.368018157 +0000 UTC m=+891.513079077" watchObservedRunningTime="2026-06-07 09:22:59.369790534 +0000 UTC m=+891.514851452" Jun 07 09:23:05.195290 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:05.195256 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:23:05.195771 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:05.195323 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:23:05.207952 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:05.207914 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:23:05.390160 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:05.390133 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6" Jun 07 09:23:08.320858 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:08.320832 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:23:08.321504 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:08.321488 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:23:36.287461 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.287421 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d8667697c-qsl4s"] Jun 07 09:23:36.290911 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.290889 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.303087 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.303060 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8667697c-qsl4s"] Jun 07 09:23:36.427173 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.427131 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/276f1b15-3120-4413-9705-856c917f77e1-tls-cert\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.427380 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.427274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl4r9\" (UniqueName: \"kubernetes.io/projected/276f1b15-3120-4413-9705-856c917f77e1-kube-api-access-pl4r9\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.427453 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.427403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/276f1b15-3120-4413-9705-856c917f77e1-oidc-ca\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.529011 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.528969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/276f1b15-3120-4413-9705-856c917f77e1-tls-cert\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.529217 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.529082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl4r9\" (UniqueName: \"kubernetes.io/projected/276f1b15-3120-4413-9705-856c917f77e1-kube-api-access-pl4r9\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.529327 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.529304 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/276f1b15-3120-4413-9705-856c917f77e1-oidc-ca\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.530136 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.530093 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/276f1b15-3120-4413-9705-856c917f77e1-oidc-ca\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.531796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.531771 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/276f1b15-3120-4413-9705-856c917f77e1-tls-cert\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.537260 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.537240 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl4r9\" (UniqueName: \"kubernetes.io/projected/276f1b15-3120-4413-9705-856c917f77e1-kube-api-access-pl4r9\") pod \"authorino-d8667697c-qsl4s\" (UID: \"276f1b15-3120-4413-9705-856c917f77e1\") " pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.604382 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.604299 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8667697c-qsl4s" Jun 07 09:23:36.739780 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:36.739757 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8667697c-qsl4s"] Jun 07 09:23:36.741986 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:23:36.741939 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod276f1b15_3120_4413_9705_856c917f77e1.slice/crio-657b71eb9acfb0e6adb79d1ced3d62554266ced1f67b033ae9337d791cbf340f WatchSource:0}: Error finding container 657b71eb9acfb0e6adb79d1ced3d62554266ced1f67b033ae9337d791cbf340f: Status 404 returned error can't find the container with id 657b71eb9acfb0e6adb79d1ced3d62554266ced1f67b033ae9337d791cbf340f Jun 07 09:23:37.548724 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.548683 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8667697c-qsl4s" event={"ID":"276f1b15-3120-4413-9705-856c917f77e1","Type":"ContainerStarted","Data":"b002e59ff381ad352091b81026e28d78fd05db8e279520d10158f4a354c0fe78"} Jun 07 09:23:37.549135 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.548738 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8667697c-qsl4s" event={"ID":"276f1b15-3120-4413-9705-856c917f77e1","Type":"ContainerStarted","Data":"657b71eb9acfb0e6adb79d1ced3d62554266ced1f67b033ae9337d791cbf340f"} Jun 07 09:23:37.564716 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.564659 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d8667697c-qsl4s" podStartSLOduration=1.113953354 podStartE2EDuration="1.564638199s" totalImagesPullingTime="450.684845ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:23:37.563865273 +0000 UTC m=+929.708926193" watchObservedRunningTime="2026-06-07 09:23:37.564638199 +0000 UTC m=+929.709699119" Jun 07 09:23:37.595066 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.595030 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-566c97dd5b-ktfvc"] Jun 07 09:23:37.595286 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.595219 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" podUID="a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" containerName="authorino" containerID="cri-o://2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2" gracePeriod=30 Jun 07 09:23:37.859429 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.859406 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:23:37.943475 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.943441 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j48v6\" (UniqueName: \"kubernetes.io/projected/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-kube-api-access-j48v6\") pod \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " Jun 07 09:23:37.943635 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.943581 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-oidc-ca\") pod \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " Jun 07 09:23:37.943635 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.943612 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-tls-cert\") pod \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\" (UID: \"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68\") " Jun 07 09:23:37.945547 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.945513 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-kube-api-access-j48v6" (OuterVolumeSpecName: "kube-api-access-j48v6") pod "a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" (UID: "a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68"). InnerVolumeSpecName "kube-api-access-j48v6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:23:37.948775 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.948752 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" (UID: "a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:23:37.954044 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:37.954020 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" (UID: "a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:23:38.045355 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.045306 2578 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-oidc-ca\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:23:38.045355 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.045336 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-tls-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:23:38.045355 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.045346 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j48v6\" (UniqueName: \"kubernetes.io/projected/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68-kube-api-access-j48v6\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:23:38.555154 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.555117 2578 generic.go:356] "Generic (PLEG): container finished" podID="a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" containerID="2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2" exitCode=0 Jun 07 09:23:38.555590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.555179 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" event={"ID":"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68","Type":"ContainerDied","Data":"2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2"} Jun 07 09:23:38.555590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.555219 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" event={"ID":"a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68","Type":"ContainerDied","Data":"3d4c503686de926cf7a05c222c46f80eb10b4008370c2f9232daa4cbda962598"} Jun 07 09:23:38.555590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.555233 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566c97dd5b-ktfvc" Jun 07 09:23:38.555590 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.555236 2578 scope.go:117] "RemoveContainer" containerID="2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2" Jun 07 09:23:38.564261 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.564209 2578 scope.go:117] "RemoveContainer" containerID="2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2" Jun 07 09:23:38.564468 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:23:38.564450 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2\": container with ID starting with 2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2 not found: ID does not exist" containerID="2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2" Jun 07 09:23:38.564523 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.564480 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2"} err="failed to get container status \"2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2\": rpc error: code = NotFound desc = could not find container \"2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2\": container with ID starting with 2dbefc910fe8192fa52ecb3e53ef1e0c35112a4db6c4fc450d815b98f3b1abf2 not found: ID does not exist" Jun 07 09:23:38.574142 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.574115 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-566c97dd5b-ktfvc"] Jun 07 09:23:38.579807 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:38.579782 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-566c97dd5b-ktfvc"] Jun 07 09:23:40.299982 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:23:40.299950 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" path="/var/lib/kubelet/pods/a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68/volumes" Jun 07 09:25:06.091742 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.091703 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-545c76b485-qcxrg"] Jun 07 09:25:06.092294 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.091981 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-545c76b485-qcxrg" podUID="49264035-b4d1-4fd7-a12c-14ae34136a03" containerName="manager" containerID="cri-o://62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45" gracePeriod=30 Jun 07 09:25:06.349871 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.349819 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:25:06.483008 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.482979 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4stq\" (UniqueName: \"kubernetes.io/projected/49264035-b4d1-4fd7-a12c-14ae34136a03-kube-api-access-j4stq\") pod \"49264035-b4d1-4fd7-a12c-14ae34136a03\" (UID: \"49264035-b4d1-4fd7-a12c-14ae34136a03\") " Jun 07 09:25:06.483165 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.483030 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49264035-b4d1-4fd7-a12c-14ae34136a03-webhook-cert\") pod \"49264035-b4d1-4fd7-a12c-14ae34136a03\" (UID: \"49264035-b4d1-4fd7-a12c-14ae34136a03\") " Jun 07 09:25:06.484916 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.484896 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49264035-b4d1-4fd7-a12c-14ae34136a03-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "49264035-b4d1-4fd7-a12c-14ae34136a03" (UID: "49264035-b4d1-4fd7-a12c-14ae34136a03"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:25:06.485004 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.484946 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49264035-b4d1-4fd7-a12c-14ae34136a03-kube-api-access-j4stq" (OuterVolumeSpecName: "kube-api-access-j4stq") pod "49264035-b4d1-4fd7-a12c-14ae34136a03" (UID: "49264035-b4d1-4fd7-a12c-14ae34136a03"). InnerVolumeSpecName "kube-api-access-j4stq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:25:06.584185 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.584156 2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49264035-b4d1-4fd7-a12c-14ae34136a03-webhook-cert\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:25:06.584185 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:06.584183 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4stq\" (UniqueName: \"kubernetes.io/projected/49264035-b4d1-4fd7-a12c-14ae34136a03-kube-api-access-j4stq\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:25:07.018633 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.018594 2578 generic.go:356] "Generic (PLEG): container finished" podID="49264035-b4d1-4fd7-a12c-14ae34136a03" containerID="62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45" exitCode=0 Jun 07 09:25:07.018810 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.018644 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-545c76b485-qcxrg" Jun 07 09:25:07.018810 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.018674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-545c76b485-qcxrg" event={"ID":"49264035-b4d1-4fd7-a12c-14ae34136a03","Type":"ContainerDied","Data":"62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45"} Jun 07 09:25:07.018810 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.018724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-545c76b485-qcxrg" event={"ID":"49264035-b4d1-4fd7-a12c-14ae34136a03","Type":"ContainerDied","Data":"77d5977ad3537b5fa5a1f31be5cfa87f978560b118fa85f70ef9ff9387170533"} Jun 07 09:25:07.018810 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.018747 2578 scope.go:117] "RemoveContainer" containerID="62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45" Jun 07 09:25:07.028271 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.028255 2578 scope.go:117] "RemoveContainer" containerID="62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45" Jun 07 09:25:07.028545 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:25:07.028527 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45\": container with ID starting with 62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45 not found: ID does not exist" containerID="62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45" Jun 07 09:25:07.028607 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.028559 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45"} err="failed to get container status \"62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45\": rpc error: code = NotFound desc = could not find container \"62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45\": container with ID starting with 62b8db211641dca41eda37180851ac1928488aaedd3cc24d9026ea768cf23e45 not found: ID does not exist" Jun 07 09:25:07.042445 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.041559 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-545c76b485-qcxrg"] Jun 07 09:25:07.050328 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.050304 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-545c76b485-qcxrg"] Jun 07 09:25:07.837418 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.837386 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-545c76b485-fh8lq"] Jun 07 09:25:07.837863 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.837844 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" containerName="authorino" Jun 07 09:25:07.837915 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.837867 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" containerName="authorino" Jun 07 09:25:07.837915 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.837889 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49264035-b4d1-4fd7-a12c-14ae34136a03" containerName="manager" Jun 07 09:25:07.837915 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.837897 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="49264035-b4d1-4fd7-a12c-14ae34136a03" containerName="manager" Jun 07 09:25:07.838057 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.838032 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="49264035-b4d1-4fd7-a12c-14ae34136a03" containerName="manager" Jun 07 09:25:07.838057 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.838048 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1898e2a-8c2c-4e41-b2c9-b1ee470b4e68" containerName="authorino" Jun 07 09:25:07.842427 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.842411 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:07.846700 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.846676 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 07 09:25:07.846807 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.846676 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 07 09:25:07.846807 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.846739 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6vrm4\"" Jun 07 09:25:07.851428 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.851405 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-545c76b485-fh8lq"] Jun 07 09:25:07.997213 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.997181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znjjc\" (UniqueName: \"kubernetes.io/projected/01d6afc4-b0f2-41c7-92db-4dec4b74ad94-kube-api-access-znjjc\") pod \"maas-controller-545c76b485-fh8lq\" (UID: \"01d6afc4-b0f2-41c7-92db-4dec4b74ad94\") " pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:07.997403 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:07.997248 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01d6afc4-b0f2-41c7-92db-4dec4b74ad94-webhook-cert\") pod \"maas-controller-545c76b485-fh8lq\" (UID: \"01d6afc4-b0f2-41c7-92db-4dec4b74ad94\") " pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:08.098525 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.098435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znjjc\" (UniqueName: \"kubernetes.io/projected/01d6afc4-b0f2-41c7-92db-4dec4b74ad94-kube-api-access-znjjc\") pod \"maas-controller-545c76b485-fh8lq\" (UID: \"01d6afc4-b0f2-41c7-92db-4dec4b74ad94\") " pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:08.098525 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.098513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01d6afc4-b0f2-41c7-92db-4dec4b74ad94-webhook-cert\") pod \"maas-controller-545c76b485-fh8lq\" (UID: \"01d6afc4-b0f2-41c7-92db-4dec4b74ad94\") " pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:08.101065 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.101036 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01d6afc4-b0f2-41c7-92db-4dec4b74ad94-webhook-cert\") pod \"maas-controller-545c76b485-fh8lq\" (UID: \"01d6afc4-b0f2-41c7-92db-4dec4b74ad94\") " pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:08.107334 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.107309 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znjjc\" (UniqueName: \"kubernetes.io/projected/01d6afc4-b0f2-41c7-92db-4dec4b74ad94-kube-api-access-znjjc\") pod \"maas-controller-545c76b485-fh8lq\" (UID: \"01d6afc4-b0f2-41c7-92db-4dec4b74ad94\") " pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:08.154221 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.154193 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:08.283872 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.283844 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-545c76b485-fh8lq"] Jun 07 09:25:08.286716 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:25:08.286687 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d6afc4_b0f2_41c7_92db_4dec4b74ad94.slice/crio-0739e2cd6e8c63986f0ee3f3409542b57dc31e062d197fc8563d8a982cbe5054 WatchSource:0}: Error finding container 0739e2cd6e8c63986f0ee3f3409542b57dc31e062d197fc8563d8a982cbe5054: Status 404 returned error can't find the container with id 0739e2cd6e8c63986f0ee3f3409542b57dc31e062d197fc8563d8a982cbe5054 Jun 07 09:25:08.288029 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.288010 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 07 09:25:08.300709 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.300680 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49264035-b4d1-4fd7-a12c-14ae34136a03" path="/var/lib/kubelet/pods/49264035-b4d1-4fd7-a12c-14ae34136a03/volumes" Jun 07 09:25:08.723457 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:08.723431 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 07 09:25:09.039799 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:09.039765 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-545c76b485-fh8lq" event={"ID":"01d6afc4-b0f2-41c7-92db-4dec4b74ad94","Type":"ContainerStarted","Data":"7f2df7f2f69a8739bee145b6c2e88f11b1995f4ee667394892c0e5760f0d3c77"} Jun 07 09:25:09.040189 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:09.039816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-545c76b485-fh8lq" event={"ID":"01d6afc4-b0f2-41c7-92db-4dec4b74ad94","Type":"ContainerStarted","Data":"0739e2cd6e8c63986f0ee3f3409542b57dc31e062d197fc8563d8a982cbe5054"} Jun 07 09:25:09.040189 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:09.039895 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:25:09.056666 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:09.056622 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-545c76b485-fh8lq" podStartSLOduration=1.6244049820000002 podStartE2EDuration="2.056604506s" totalImagesPullingTime="432.199524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:25:09.055481157 +0000 UTC m=+1021.200542077" watchObservedRunningTime="2026-06-07 09:25:09.056604506 +0000 UTC m=+1021.201665425" Jun 07 09:25:20.050699 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:25:20.050666 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-545c76b485-fh8lq" Jun 07 09:28:08.353149 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:28:08.353119 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:28:08.355245 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:28:08.355225 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:33:08.392512 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:33:08.392483 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:33:08.394591 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:33:08.394565 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:35:34.030416 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.030377 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk"] Jun 07 09:35:34.032388 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.030582 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" podUID="3a9c326f-059c-4376-8b66-8004204491b1" containerName="manager" containerID="cri-o://a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1" gracePeriod=10 Jun 07 09:35:34.288345 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.288282 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:35:34.313684 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.313650 2578 generic.go:356] "Generic (PLEG): container finished" podID="3a9c326f-059c-4376-8b66-8004204491b1" containerID="a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1" exitCode=0 Jun 07 09:35:34.313854 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.313710 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" Jun 07 09:35:34.313854 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.313821 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" event={"ID":"3a9c326f-059c-4376-8b66-8004204491b1","Type":"ContainerDied","Data":"a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1"} Jun 07 09:35:34.314026 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.313868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk" event={"ID":"3a9c326f-059c-4376-8b66-8004204491b1","Type":"ContainerDied","Data":"ecb925e036c6ca124722bba01f1904a0330538a0d4725e7d815fd78c8fc77b2f"} Jun 07 09:35:34.314026 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.313900 2578 scope.go:117] "RemoveContainer" containerID="a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1" Jun 07 09:35:34.325462 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.325438 2578 scope.go:117] "RemoveContainer" containerID="a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1" Jun 07 09:35:34.325758 ip-10-0-129-203 kubenswrapper[2578]: E0607 09:35:34.325729 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1\": container with ID starting with a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1 not found: ID does not exist" containerID="a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1" Jun 07 09:35:34.325819 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.325773 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1"} err="failed to get container status \"a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1\": rpc error: code = NotFound desc = could not find container \"a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1\": container with ID starting with a81ed6f5d7544df3ad380a916541cf661385a3dc585198178951aae54be9efd1 not found: ID does not exist" Jun 07 09:35:34.423796 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.423758 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a9c326f-059c-4376-8b66-8004204491b1-extensions-socket-volume\") pod \"3a9c326f-059c-4376-8b66-8004204491b1\" (UID: \"3a9c326f-059c-4376-8b66-8004204491b1\") " Jun 07 09:35:34.424016 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.423877 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp474\" (UniqueName: \"kubernetes.io/projected/3a9c326f-059c-4376-8b66-8004204491b1-kube-api-access-rp474\") pod \"3a9c326f-059c-4376-8b66-8004204491b1\" (UID: \"3a9c326f-059c-4376-8b66-8004204491b1\") " Jun 07 09:35:34.424180 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.424151 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a9c326f-059c-4376-8b66-8004204491b1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3a9c326f-059c-4376-8b66-8004204491b1" (UID: "3a9c326f-059c-4376-8b66-8004204491b1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:35:34.424270 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.424253 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3a9c326f-059c-4376-8b66-8004204491b1-extensions-socket-volume\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:35:34.425967 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.425950 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a9c326f-059c-4376-8b66-8004204491b1-kube-api-access-rp474" (OuterVolumeSpecName: "kube-api-access-rp474") pod "3a9c326f-059c-4376-8b66-8004204491b1" (UID: "3a9c326f-059c-4376-8b66-8004204491b1"). InnerVolumeSpecName "kube-api-access-rp474". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:35:34.525451 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.525409 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp474\" (UniqueName: \"kubernetes.io/projected/3a9c326f-059c-4376-8b66-8004204491b1-kube-api-access-rp474\") on node \"ip-10-0-129-203.ec2.internal\" DevicePath \"\"" Jun 07 09:35:34.636633 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.636602 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk"] Jun 07 09:35:34.639811 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:34.639788 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cl5rk"] Jun 07 09:35:36.300524 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:35:36.300486 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a9c326f-059c-4376-8b66-8004204491b1" path="/var/lib/kubelet/pods/3a9c326f-059c-4376-8b66-8004204491b1/volumes" Jun 07 09:36:40.144882 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.144670 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb"] Jun 07 09:36:40.145747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.145721 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a9c326f-059c-4376-8b66-8004204491b1" containerName="manager" Jun 07 09:36:40.145747 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.145749 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a9c326f-059c-4376-8b66-8004204491b1" containerName="manager" Jun 07 09:36:40.146079 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.146061 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a9c326f-059c-4376-8b66-8004204491b1" containerName="manager" Jun 07 09:36:40.150411 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.150384 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:40.153532 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.153510 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4ljbr\"" Jun 07 09:36:40.158244 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.158218 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb"] Jun 07 09:36:40.234749 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.234715 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd55s\" (UniqueName: \"kubernetes.io/projected/3b194ee1-2d94-46ea-adf4-816ac1cc3669-kube-api-access-nd55s\") pod \"kuadrant-operator-controller-manager-86878c769f-zj7jb\" (UID: \"3b194ee1-2d94-46ea-adf4-816ac1cc3669\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:40.234749 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.234759 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b194ee1-2d94-46ea-adf4-816ac1cc3669-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zj7jb\" (UID: \"3b194ee1-2d94-46ea-adf4-816ac1cc3669\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:40.335799 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.335744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd55s\" (UniqueName: \"kubernetes.io/projected/3b194ee1-2d94-46ea-adf4-816ac1cc3669-kube-api-access-nd55s\") pod \"kuadrant-operator-controller-manager-86878c769f-zj7jb\" (UID: \"3b194ee1-2d94-46ea-adf4-816ac1cc3669\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:40.336048 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.335811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b194ee1-2d94-46ea-adf4-816ac1cc3669-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zj7jb\" (UID: \"3b194ee1-2d94-46ea-adf4-816ac1cc3669\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:40.336251 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.336232 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b194ee1-2d94-46ea-adf4-816ac1cc3669-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zj7jb\" (UID: \"3b194ee1-2d94-46ea-adf4-816ac1cc3669\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:40.347705 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.347675 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd55s\" (UniqueName: \"kubernetes.io/projected/3b194ee1-2d94-46ea-adf4-816ac1cc3669-kube-api-access-nd55s\") pod \"kuadrant-operator-controller-manager-86878c769f-zj7jb\" (UID: \"3b194ee1-2d94-46ea-adf4-816ac1cc3669\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:40.462974 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.462850 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:40.605759 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.605730 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb"] Jun 07 09:36:40.607358 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:36:40.607317 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b194ee1_2d94_46ea_adf4_816ac1cc3669.slice/crio-8a5829f47f81f1eb72bfff9ff2adb7725e2b10dab5287a0b2881cb71fa27e5d3 WatchSource:0}: Error finding container 8a5829f47f81f1eb72bfff9ff2adb7725e2b10dab5287a0b2881cb71fa27e5d3: Status 404 returned error can't find the container with id 8a5829f47f81f1eb72bfff9ff2adb7725e2b10dab5287a0b2881cb71fa27e5d3 Jun 07 09:36:40.656184 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:40.656156 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" event={"ID":"3b194ee1-2d94-46ea-adf4-816ac1cc3669","Type":"ContainerStarted","Data":"8a5829f47f81f1eb72bfff9ff2adb7725e2b10dab5287a0b2881cb71fa27e5d3"} Jun 07 09:36:41.663634 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:41.663597 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" event={"ID":"3b194ee1-2d94-46ea-adf4-816ac1cc3669","Type":"ContainerStarted","Data":"68a0339033d8451e6dd443fd1d06c80f89f54ab055155a760746c620a805263b"} Jun 07 09:36:41.664033 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:41.663725 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:36:41.697811 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:41.697745 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" podStartSLOduration=1.6977267889999998 podStartE2EDuration="1.697726789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:36:41.694350963 +0000 UTC m=+1713.839411882" watchObservedRunningTime="2026-06-07 09:36:41.697726789 +0000 UTC m=+1713.842787707" Jun 07 09:36:52.670766 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:36:52.670735 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zj7jb" Jun 07 09:38:08.426784 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:38:08.426757 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:38:08.429886 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:38:08.429854 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:43:08.466847 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:43:08.466817 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:43:08.469795 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:43:08.469773 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:46:44.687998 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:44.687967 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d8667697c-qsl4s_276f1b15-3120-4413-9705-856c917f77e1/authorino/0.log" Jun 07 09:46:49.525285 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:49.525254 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-d8cd54b68-wdxsm_43be9139-5ccf-4095-a0da-2b565c0e0f3b/maas-api/0.log" Jun 07 09:46:49.850677 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:49.850593 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-545c76b485-fh8lq_01d6afc4-b0f2-41c7-92db-4dec4b74ad94/manager/0.log" Jun 07 09:46:50.432774 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:50.432748 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c76964bf-xbx77_206da481-8d6c-42d7-aa37-60ad403bc84e/manager/0.log" Jun 07 09:46:50.437558 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:50.437531 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c76964bf-xbx77_206da481-8d6c-42d7-aa37-60ad403bc84e/copy-manifests/0.log" Jun 07 09:46:51.742259 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:51.742226 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d8667697c-qsl4s_276f1b15-3120-4413-9705-856c917f77e1/authorino/0.log" Jun 07 09:46:52.332779 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:52.332721 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zj7jb_3b194ee1-2d94-46ea-adf4-816ac1cc3669/manager/0.log" Jun 07 09:46:53.235324 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:53.235295 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-vcrkm_982eacff-74c5-443b-8c38-38c530a3aae2/kube-auth-proxy/0.log" Jun 07 09:46:53.463202 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:53.463170 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-wch7k_36a07749-c4c8-4bf0-baad-9124f99d7a90/payload-pre-processing/0.log" Jun 07 09:46:53.575944 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:53.575829 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-jgjrm_7ffe3526-707c-4820-8330-b02bb20821c9/payload-processing/0.log" Jun 07 09:46:54.011674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:54.011647 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm_4e483922-3adb-4390-aed9-ce9d54b774d8/storage-initializer/0.log" Jun 07 09:46:54.018225 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:54.018206 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-8s9pm_4e483922-3adb-4390-aed9-ce9d54b774d8/main/0.log" Jun 07 09:46:54.357550 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:54.357477 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht_b9b5a335-503d-4b2e-b6ce-b2561ee0ab74/storage-initializer/0.log" Jun 07 09:46:54.363281 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:54.363265 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477clqvht_b9b5a335-503d-4b2e-b6ce-b2561ee0ab74/main/0.log" Jun 07 09:46:54.468971 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:54.468919 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6_c4090155-f683-424e-aa9b-ca774bccfb4b/storage-initializer/0.log" Jun 07 09:46:54.475146 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:46:54.475131 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-2zzf6_c4090155-f683-424e-aa9b-ca774bccfb4b/main/0.log" Jun 07 09:47:01.179660 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:01.179630 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n4kfk_e38f9e64-890b-4696-99fd-2ca38d35a05d/global-pull-secret-syncer/0.log" Jun 07 09:47:01.342435 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:01.342409 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xbns8_7102b2e3-1052-46e4-8f26-e9da44934255/konnectivity-agent/0.log" Jun 07 09:47:01.383829 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:01.383800 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-203.ec2.internal_62ac29cdc11a60617fbf64ef64f4afc5/haproxy/0.log" Jun 07 09:47:05.596366 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:05.596328 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d8667697c-qsl4s_276f1b15-3120-4413-9705-856c917f77e1/authorino/0.log" Jun 07 09:47:05.816569 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:05.816535 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zj7jb_3b194ee1-2d94-46ea-adf4-816ac1cc3669/manager/0.log" Jun 07 09:47:08.034884 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.034850 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-kkw42_a00eb047-24f1-4eb0-8a2d-cf79c57e4cd4/cluster-monitoring-operator/0.log" Jun 07 09:47:08.058763 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.058739 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-lg895_69fe17f9-556d-4f5c-9965-c4aa5a17357c/kube-state-metrics/0.log" Jun 07 09:47:08.079845 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.079823 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-lg895_69fe17f9-556d-4f5c-9965-c4aa5a17357c/kube-rbac-proxy-main/0.log" Jun 07 09:47:08.104407 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.104387 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-lg895_69fe17f9-556d-4f5c-9965-c4aa5a17357c/kube-rbac-proxy-self/0.log" Jun 07 09:47:08.345482 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.345451 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-td5nj_b995a55a-31f5-40d9-b1e5-7bb42213e984/node-exporter/0.log" Jun 07 09:47:08.366794 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.366769 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-td5nj_b995a55a-31f5-40d9-b1e5-7bb42213e984/kube-rbac-proxy/0.log" Jun 07 09:47:08.391630 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.391610 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-td5nj_b995a55a-31f5-40d9-b1e5-7bb42213e984/init-textfile/0.log" Jun 07 09:47:08.511620 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.511596 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-lvr2h_36f1f148-57cb-411b-9590-66af20af1dd3/kube-rbac-proxy-main/0.log" Jun 07 09:47:08.537466 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.537426 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-lvr2h_36f1f148-57cb-411b-9590-66af20af1dd3/kube-rbac-proxy-self/0.log" Jun 07 09:47:08.568635 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.568565 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-lvr2h_36f1f148-57cb-411b-9590-66af20af1dd3/openshift-state-metrics/0.log" Jun 07 09:47:08.611000 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.610970 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_440b2bb3-6007-4795-9f02-5ec72cc97922/prometheus/0.log" Jun 07 09:47:08.640281 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.640259 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_440b2bb3-6007-4795-9f02-5ec72cc97922/config-reloader/0.log" Jun 07 09:47:08.662837 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.662798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_440b2bb3-6007-4795-9f02-5ec72cc97922/thanos-sidecar/0.log" Jun 07 09:47:08.686430 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.686413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_440b2bb3-6007-4795-9f02-5ec72cc97922/kube-rbac-proxy-web/0.log" Jun 07 09:47:08.710832 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.710797 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_440b2bb3-6007-4795-9f02-5ec72cc97922/kube-rbac-proxy/0.log" Jun 07 09:47:08.734413 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.734384 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_440b2bb3-6007-4795-9f02-5ec72cc97922/kube-rbac-proxy-thanos/0.log" Jun 07 09:47:08.762654 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.762636 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_440b2bb3-6007-4795-9f02-5ec72cc97922/init-config-reloader/0.log" Jun 07 09:47:08.796992 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.796966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-lnxf9_73c0e393-a1fa-4493-b98c-4f729e7a2133/prometheus-operator/0.log" Jun 07 09:47:08.823024 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:08.822943 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-lnxf9_73c0e393-a1fa-4493-b98c-4f729e7a2133/kube-rbac-proxy/0.log" Jun 07 09:47:09.057365 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:09.057337 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7694495584-gw8c4_beada5f8-a52b-4b4f-af31-7610d98fd074/thanos-query/0.log" Jun 07 09:47:09.083785 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:09.083719 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7694495584-gw8c4_beada5f8-a52b-4b4f-af31-7610d98fd074/kube-rbac-proxy-web/0.log" Jun 07 09:47:09.106866 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:09.106844 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7694495584-gw8c4_beada5f8-a52b-4b4f-af31-7610d98fd074/kube-rbac-proxy/0.log" Jun 07 09:47:09.131407 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:09.131382 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7694495584-gw8c4_beada5f8-a52b-4b4f-af31-7610d98fd074/prom-label-proxy/0.log" Jun 07 09:47:09.155632 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:09.155611 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7694495584-gw8c4_beada5f8-a52b-4b4f-af31-7610d98fd074/kube-rbac-proxy-rules/0.log" Jun 07 09:47:09.181593 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:09.181569 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7694495584-gw8c4_beada5f8-a52b-4b4f-af31-7610d98fd074/kube-rbac-proxy-metrics/0.log" Jun 07 09:47:12.067253 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:12.067228 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-lqxf6_657f9a89-a56d-4317-8197-9367dcb37cc8/download-server/0.log" Jun 07 09:47:12.785746 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:12.785725 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-fcrnd_a58d20d0-9c6c-48d6-b3f8-52cd282b086f/volume-data-source-validator/0.log" Jun 07 09:47:13.478826 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.478794 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw"] Jun 07 09:47:13.482080 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.482052 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.484647 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.484626 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jfwxs\"/\"openshift-service-ca.crt\"" Jun 07 09:47:13.485985 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.485962 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jfwxs\"/\"kube-root-ca.crt\"" Jun 07 09:47:13.486079 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.486006 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jfwxs\"/\"default-dockercfg-nh8jc\"" Jun 07 09:47:13.492144 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.492120 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw"] Jun 07 09:47:13.570988 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.570963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-sys\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.571141 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.571021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-lib-modules\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.571141 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.571071 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-podres\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.571141 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.571108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-proc\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.571247 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.571161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g2jl\" (UniqueName: \"kubernetes.io/projected/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-kube-api-access-8g2jl\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672429 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-sys\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672571 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-lib-modules\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672571 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-podres\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672571 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672523 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-sys\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672571 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-proc\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672571 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672562 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-proc\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672763 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672580 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-lib-modules\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672763 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672611 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g2jl\" (UniqueName: \"kubernetes.io/projected/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-kube-api-access-8g2jl\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.672763 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.672653 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-podres\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.680375 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.680343 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g2jl\" (UniqueName: \"kubernetes.io/projected/a76937e2-90b1-4fe0-a8bb-60ccdce0af6a-kube-api-access-8g2jl\") pod \"perf-node-gather-daemonset-4knkw\" (UID: \"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.793311 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.793239 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:13.924208 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:13.924186 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw"] Jun 07 09:47:13.925737 ip-10-0-129-203 kubenswrapper[2578]: W0607 09:47:13.925698 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda76937e2_90b1_4fe0_a8bb_60ccdce0af6a.slice/crio-2a1985a4cd7bd28fa4f36d842334bda5beb565cba645b271214681279fb54582 WatchSource:0}: Error finding container 2a1985a4cd7bd28fa4f36d842334bda5beb565cba645b271214681279fb54582: Status 404 returned error can't find the container with id 2a1985a4cd7bd28fa4f36d842334bda5beb565cba645b271214681279fb54582 Jun 07 09:47:14.012630 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:14.012592 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gqbxw_fa52f142-0b4a-40a4-a7e3-3445e168731a/dns/0.log" Jun 07 09:47:14.031012 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:14.030973 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gqbxw_fa52f142-0b4a-40a4-a7e3-3445e168731a/kube-rbac-proxy/0.log" Jun 07 09:47:14.105351 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:14.105280 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dfktq_d7f74ed1-5a63-4261-b482-e4a9e2b2e47b/dns-node-resolver/0.log" Jun 07 09:47:14.719942 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:14.719897 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-msxgz_795d8080-8304-4fe8-822b-723fc3feafbe/node-ca/0.log" Jun 07 09:47:14.908732 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:14.908697 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" event={"ID":"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a","Type":"ContainerStarted","Data":"3a8619d2635c9d23c5046a42d31677485148e31cefa06ed1db5bf73c1b2d4515"} Jun 07 09:47:14.908913 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:14.908755 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" event={"ID":"a76937e2-90b1-4fe0-a8bb-60ccdce0af6a","Type":"ContainerStarted","Data":"2a1985a4cd7bd28fa4f36d842334bda5beb565cba645b271214681279fb54582"} Jun 07 09:47:14.908913 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:14.908792 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:14.925331 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:14.925283 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" podStartSLOduration=1.925270016 podStartE2EDuration="1.925270016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:47:14.923019671 +0000 UTC m=+2347.068080647" watchObservedRunningTime="2026-06-07 09:47:14.925270016 +0000 UTC m=+2347.070330934" Jun 07 09:47:15.821149 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:15.821117 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-vcrkm_982eacff-74c5-443b-8c38-38c530a3aae2/kube-auth-proxy/0.log" Jun 07 09:47:15.948245 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:15.948217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-wch7k_36a07749-c4c8-4bf0-baad-9124f99d7a90/payload-pre-processing/0.log" Jun 07 09:47:16.004546 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:16.004517 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-jgjrm_7ffe3526-707c-4820-8330-b02bb20821c9/payload-processing/0.log" Jun 07 09:47:16.693598 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:16.693569 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gmnbx_dc36ff62-339f-4ea0-92fb-3f15cfebdd00/serve-healthcheck-canary/0.log" Jun 07 09:47:17.189718 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:17.189664 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-vf2x8_61c5adf4-9ecc-4355-a255-2aa644c130b9/insights-operator/0.log" Jun 07 09:47:17.190882 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:17.190865 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-vf2x8_61c5adf4-9ecc-4355-a255-2aa644c130b9/insights-operator/1.log" Jun 07 09:47:17.268400 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:17.268363 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njb2z_ba660a44-ade6-4b34-99a6-229bb1872072/kube-rbac-proxy/0.log" Jun 07 09:47:17.287191 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:17.287169 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njb2z_ba660a44-ade6-4b34-99a6-229bb1872072/exporter/0.log" Jun 07 09:47:17.305955 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:17.305916 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-njb2z_ba660a44-ade6-4b34-99a6-229bb1872072/extractor/0.log" Jun 07 09:47:19.712724 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:19.712690 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-d8cd54b68-wdxsm_43be9139-5ccf-4095-a0da-2b565c0e0f3b/maas-api/0.log" Jun 07 09:47:19.828071 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:19.828037 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-545c76b485-fh8lq_01d6afc4-b0f2-41c7-92db-4dec4b74ad94/manager/0.log" Jun 07 09:47:20.193999 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:20.193966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c76964bf-xbx77_206da481-8d6c-42d7-aa37-60ad403bc84e/manager/0.log" Jun 07 09:47:20.211211 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:20.211187 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c76964bf-xbx77_206da481-8d6c-42d7-aa37-60ad403bc84e/copy-manifests/0.log" Jun 07 09:47:20.923530 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:20.923500 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-4knkw" Jun 07 09:47:21.709135 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:21.709110 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-867d454bbd-2gc9m_18c76e8d-33c8-442c-ac2a-dc83407c0278/manager/0.log" Jun 07 09:47:26.303284 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:26.303254 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-vcrkm_982eacff-74c5-443b-8c38-38c530a3aae2/kube-auth-proxy/0.log" Jun 07 09:47:26.417692 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:26.417664 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-wch7k_36a07749-c4c8-4bf0-baad-9124f99d7a90/payload-pre-processing/0.log" Jun 07 09:47:26.473720 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:26.473693 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-jgjrm_7ffe3526-707c-4820-8330-b02bb20821c9/payload-processing/0.log" Jun 07 09:47:27.220655 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:27.220617 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-pqqqk_ca89a42b-4d21-4d22-9181-e72d02a56659/migrator/0.log" Jun 07 09:47:27.245512 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:27.245486 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-pqqqk_ca89a42b-4d21-4d22-9181-e72d02a56659/graceful-termination/0.log" Jun 07 09:47:27.640077 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:27.640045 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-qw7sk_da36a86e-4fab-4dad-85e3-0c9232d4a5f0/kube-storage-version-migrator-operator/1.log" Jun 07 09:47:27.640904 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:27.640884 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-qw7sk_da36a86e-4fab-4dad-85e3-0c9232d4a5f0/kube-storage-version-migrator-operator/0.log" Jun 07 09:47:27.686158 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:27.686133 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-vcrkm_982eacff-74c5-443b-8c38-38c530a3aae2/kube-auth-proxy/0.log" Jun 07 09:47:27.808211 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:27.808181 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-wch7k_36a07749-c4c8-4bf0-baad-9124f99d7a90/payload-pre-processing/0.log" Jun 07 09:47:27.864007 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:27.863979 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-jgjrm_7ffe3526-707c-4820-8330-b02bb20821c9/payload-processing/0.log" Jun 07 09:47:28.883182 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:28.883157 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dvckp_8904cd6d-8eaf-4852-8672-9e1651cab35f/kube-multus-additional-cni-plugins/0.log" Jun 07 09:47:28.906432 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:28.906408 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dvckp_8904cd6d-8eaf-4852-8672-9e1651cab35f/egress-router-binary-copy/0.log" Jun 07 09:47:28.929227 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:28.929209 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dvckp_8904cd6d-8eaf-4852-8672-9e1651cab35f/cni-plugins/0.log" Jun 07 09:47:28.958462 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:28.958440 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dvckp_8904cd6d-8eaf-4852-8672-9e1651cab35f/bond-cni-plugin/0.log" Jun 07 09:47:28.981993 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:28.981971 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dvckp_8904cd6d-8eaf-4852-8672-9e1651cab35f/routeoverride-cni/0.log" Jun 07 09:47:29.003213 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:29.003190 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dvckp_8904cd6d-8eaf-4852-8672-9e1651cab35f/whereabouts-cni-bincopy/0.log" Jun 07 09:47:29.023855 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:29.023836 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dvckp_8904cd6d-8eaf-4852-8672-9e1651cab35f/whereabouts-cni/0.log" Jun 07 09:47:29.368847 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:29.368813 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cv9fw_e47627b3-c7bf-4e90-9ee8-69919c6b8c13/kube-multus/0.log" Jun 07 09:47:29.503217 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:29.503196 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p4shp_dfbdde5e-d359-422e-bf75-55f3221bd8f3/network-metrics-daemon/0.log" Jun 07 09:47:29.519674 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:29.519654 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-p4shp_dfbdde5e-d359-422e-bf75-55f3221bd8f3/kube-rbac-proxy/0.log" Jun 07 09:47:29.713946 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:29.713857 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-vcrkm_982eacff-74c5-443b-8c38-38c530a3aae2/kube-auth-proxy/0.log" Jun 07 09:47:29.828227 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:29.828183 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-wch7k_36a07749-c4c8-4bf0-baad-9124f99d7a90/payload-pre-processing/0.log" Jun 07 09:47:29.874543 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:29.874518 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-jgjrm_7ffe3526-707c-4820-8330-b02bb20821c9/payload-processing/0.log" Jun 07 09:47:30.713127 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.713102 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-controller/0.log" Jun 07 09:47:30.729412 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.729388 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/1.log" Jun 07 09:47:30.729553 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.729535 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovn-acl-logging/0.log" Jun 07 09:47:30.746142 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.746122 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/kube-rbac-proxy-node/0.log" Jun 07 09:47:30.764517 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.764490 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/kube-rbac-proxy-ovn-metrics/0.log" Jun 07 09:47:30.779745 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.779728 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/northd/0.log" Jun 07 09:47:30.798322 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.798298 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/nbdb/0.log" Jun 07 09:47:30.816318 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.816293 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/sbdb/0.log" Jun 07 09:47:30.865103 ip-10-0-129-203 kubenswrapper[2578]: I0607 09:47:30.865081 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6tljf_d5cd375f-3dde-4e82-a91e-fe1b006c0cc9/ovnkube-controller/0.log"