Jun 11 14:56:11.093186 ip-10-0-140-77 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 11 14:56:11.093203 ip-10-0-140-77 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 11 14:56:11.093213 ip-10-0-140-77 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 11 14:56:11.093373 ip-10-0-140-77 systemd[1]: Failed to start Kubernetes Kubelet. Jun 11 14:56:21.143030 ip-10-0-140-77 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 11 14:56:21.143047 ip-10-0-140-77 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5b00e365a93c4a589fd9380dd7fbdcac -- Jun 11 14:59:03.102059 ip-10-0-140-77 systemd[1]: Starting Kubernetes Kubelet... Jun 11 14:59:03.518705 ip-10-0-140-77 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 14:59:03.518705 ip-10-0-140-77 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 14:59:03.518705 ip-10-0-140-77 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 14:59:03.518705 ip-10-0-140-77 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 14:59:03.518705 ip-10-0-140-77 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 14:59:03.520856 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.520631 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 14:59:03.525175 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525159 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 14:59:03.525175 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525174 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525178 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525182 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525185 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525188 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525191 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525195 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525198 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525202 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525206 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525215 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525218 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525222 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525225 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525228 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525231 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525234 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525237 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525240 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 14:59:03.525248 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525243 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525245 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525248 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525250 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525253 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525256 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525259 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525262 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525264 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525267 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525269 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525273 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525275 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525278 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525280 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525283 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525285 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525288 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525290 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525293 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 14:59:03.525705 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525295 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525298 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525300 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525303 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525305 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525308 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525311 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525313 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525316 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525318 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525321 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525323 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525326 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525328 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525331 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525334 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525337 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525339 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525342 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525344 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 14:59:03.526206 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525347 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525349 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525351 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525354 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525356 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525359 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525361 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525364 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525366 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525369 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525371 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525374 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525377 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525379 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525382 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525384 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525387 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525389 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525394 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525397 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 14:59:03.526684 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525399 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525401 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525404 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525406 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525409 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525411 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525415 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525418 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525421 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525423 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525426 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525428 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525431 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525433 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525436 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525438 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525441 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525443 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.525446 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526162 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 14:59:03.527209 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526168 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526171 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526174 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526177 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526181 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526183 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526186 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526189 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526193 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526195 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526198 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526201 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526204 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526206 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526209 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526211 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526215 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526218 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526221 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526225 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 14:59:03.527699 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526227 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526230 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526233 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526235 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526239 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526243 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526246 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526249 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526252 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526255 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526257 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526260 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526263 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526266 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526268 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526270 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526273 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526276 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526279 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 14:59:03.528213 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526281 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526285 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526289 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526291 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526294 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526297 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526300 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526303 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526305 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526308 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526311 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526314 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526317 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526320 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526322 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526325 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526327 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526330 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526332 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 14:59:03.528668 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526335 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526338 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526340 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526343 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526347 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526350 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526352 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526355 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526357 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526360 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526363 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526367 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526370 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526373 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526375 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526378 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526381 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526384 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526386 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526389 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 14:59:03.529229 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526391 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526394 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526396 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526399 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526402 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526404 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526407 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526409 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526411 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526414 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526417 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526419 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526422 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526425 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526427 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526430 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526432 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526434 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526437 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526439 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 14:59:03.529728 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526814 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526819 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526822 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526825 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526829 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526832 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526835 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526838 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526841 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526843 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526846 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526848 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526851 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526854 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526856 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526860 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526862 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526865 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526868 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526870 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 14:59:03.530215 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526873 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526876 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526880 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526883 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526886 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526889 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526891 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526894 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526896 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526899 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526901 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526904 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526906 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526909 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526911 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526914 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526916 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526920 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526922 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 14:59:03.530702 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526925 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526927 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526930 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526932 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526938 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526941 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526943 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526946 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526949 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526951 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526954 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526956 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526959 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526962 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526964 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526967 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526969 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526971 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526974 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 14:59:03.531173 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526976 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526981 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526984 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526987 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526989 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526992 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526995 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.526998 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527001 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527003 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527006 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527009 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527012 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527015 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527018 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527021 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527024 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527028 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527031 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 14:59:03.531648 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527033 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527037 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527040 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527042 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527045 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527047 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527050 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527053 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527055 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527059 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527063 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527066 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527069 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527071 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527074 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527077 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527080 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527082 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527085 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527088 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 14:59:03.532157 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527091 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527093 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527167 2562 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527177 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527186 2562 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527192 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527200 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527203 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527208 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527212 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527217 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527220 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527224 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527227 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527231 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527234 2562 flags.go:64] FLAG: --cgroup-root="" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527237 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527240 2562 flags.go:64] FLAG: --client-ca-file="" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527243 2562 flags.go:64] FLAG: --cloud-provider="external" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527246 2562 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527250 2562 flags.go:64] FLAG: --cluster-domain="" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527253 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527256 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527259 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 14:59:03.532632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527263 2562 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527266 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527269 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527272 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527276 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527279 2562 flags.go:64] FLAG: --contention-profiling="false" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527282 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527285 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527288 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527291 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527295 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527298 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527301 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527304 2562 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527307 2562 flags.go:64] FLAG: --enable-server="true" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527310 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527314 2562 flags.go:64] FLAG: --event-burst="100" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527317 2562 flags.go:64] FLAG: --event-qps="50" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527325 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527328 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527331 2562 flags.go:64] FLAG: --eviction-hard="" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527335 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527341 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527343 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527346 2562 flags.go:64] FLAG: --eviction-soft="" Jun 11 14:59:03.533250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527350 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527353 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527356 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527359 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527362 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527364 2562 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527367 2562 flags.go:64] FLAG: --feature-gates="" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527371 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527374 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527377 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527380 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527383 2562 flags.go:64] FLAG: --healthz-port="10248" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527386 2562 flags.go:64] FLAG: --help="false" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527389 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527392 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527395 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527398 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527401 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527405 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527407 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527410 2562 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527413 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527416 2562 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 14:59:03.533904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527419 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527423 2562 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527427 2562 flags.go:64] FLAG: --kube-reserved="" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527430 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527432 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527436 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527440 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527442 2562 flags.go:64] FLAG: --lock-file="" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527445 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527448 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527451 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527457 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527460 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527463 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527465 2562 flags.go:64] FLAG: --logging-format="text" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527468 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527471 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527474 2562 flags.go:64] FLAG: --manifest-url="" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527477 2562 flags.go:64] FLAG: --manifest-url-header="" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527482 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527485 2562 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527489 2562 flags.go:64] FLAG: --max-pods="110" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527492 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527494 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527505 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 14:59:03.534937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527508 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527511 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527514 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527517 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527524 2562 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527528 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527531 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527534 2562 flags.go:64] FLAG: --pod-cidr="" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527537 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527545 2562 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527548 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527551 2562 flags.go:64] FLAG: --pods-per-core="0" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527554 2562 flags.go:64] FLAG: --port="10250" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527557 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527562 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cbe4d01154c82c33" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527565 2562 flags.go:64] FLAG: --qos-reserved="" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527568 2562 flags.go:64] FLAG: --read-only-port="10255" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527571 2562 flags.go:64] FLAG: --register-node="true" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527574 2562 flags.go:64] FLAG: --register-with-taints="" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527577 2562 flags.go:64] FLAG: --registry-burst="10" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527580 2562 flags.go:64] FLAG: --registry-qps="5" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527583 2562 flags.go:64] FLAG: --reserved-cpus="" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527586 2562 flags.go:64] FLAG: --reserved-memory="" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527590 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527593 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 14:59:03.536061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527596 2562 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527599 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527602 2562 flags.go:64] FLAG: --runonce="false" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527605 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527608 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527611 2562 flags.go:64] FLAG: --seccomp-default="false" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527614 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527617 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527620 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527623 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527626 2562 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527629 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527632 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527634 2562 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527637 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527640 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527645 2562 flags.go:64] FLAG: --system-cgroups="" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527648 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527653 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527656 2562 flags.go:64] FLAG: --tls-cert-file="" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527659 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527665 2562 flags.go:64] FLAG: --tls-min-version="" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527667 2562 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527670 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527673 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 14:59:03.536861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527676 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527679 2562 flags.go:64] FLAG: --v="2" Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527682 2562 flags.go:64] FLAG: --version="false" Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527686 2562 flags.go:64] FLAG: --vmodule="" Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527691 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.527694 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527817 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527821 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527824 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527827 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527829 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527832 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527834 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527838 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527842 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527845 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527848 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527851 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527853 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527856 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 14:59:03.537559 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527858 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527861 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527864 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527867 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527870 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527873 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527875 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527878 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527882 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527885 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527887 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527890 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527892 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527894 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527897 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527899 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527902 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527905 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527908 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527910 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 14:59:03.538493 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527913 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527915 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527918 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527921 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527925 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527928 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527931 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527933 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527936 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527939 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527941 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527944 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527946 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527949 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527951 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527954 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527957 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527960 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527963 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 14:59:03.539151 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527966 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527970 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527974 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527977 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527980 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527983 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527986 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527989 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527991 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527994 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527996 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.527999 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528002 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528004 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528007 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528010 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528012 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528015 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528017 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528020 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 14:59:03.539900 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528022 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528025 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528027 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528030 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528032 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528035 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528037 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528040 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528043 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528045 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528048 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528051 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528054 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528057 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528060 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528063 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528065 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528068 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528070 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528074 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 14:59:03.540835 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528076 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528079 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528081 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528084 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528086 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.528089 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.528793 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.536213 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.536232 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.537364 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.537379 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537466 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537473 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537479 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 14:59:03.541392 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537484 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537492 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537498 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537503 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537508 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537513 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537518 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537522 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537527 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537532 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537536 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537541 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537546 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537550 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537554 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537558 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537562 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537567 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537570 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537575 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 14:59:03.541780 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537579 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537583 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537588 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537592 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537596 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537600 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537607 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537613 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537617 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537621 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537626 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537630 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537634 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537639 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537643 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537647 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537651 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537655 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537660 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537664 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 14:59:03.542269 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537668 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537672 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537676 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537681 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537685 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537690 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537694 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537698 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537703 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537707 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537727 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537732 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537737 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537741 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537745 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537750 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537754 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537759 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537763 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537769 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 14:59:03.542755 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537773 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537777 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537782 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537786 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537791 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537795 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537802 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537807 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537812 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537817 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537822 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537826 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537830 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537834 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537839 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537844 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537848 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537852 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537857 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 14:59:03.543251 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537861 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537865 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537870 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537874 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537879 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537883 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537887 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537891 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537898 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537903 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537908 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537914 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537919 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537924 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537929 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537934 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 14:59:03.543731 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.537938 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.537946 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538134 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538142 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538147 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538151 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538155 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538159 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538163 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538168 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538173 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538177 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538181 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538185 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538190 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 14:59:03.544123 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538194 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538198 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538203 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538207 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538211 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538215 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538220 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538224 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538228 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538233 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538237 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538241 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538245 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538251 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538255 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538260 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538264 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538269 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538273 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538277 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 14:59:03.544492 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538281 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538285 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538291 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538299 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538305 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538309 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538314 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538319 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538325 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538331 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538336 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538341 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538346 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538350 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538355 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538359 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538363 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538367 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538371 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 14:59:03.544992 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538376 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538380 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538385 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538389 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538393 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538398 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538403 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538407 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538411 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538416 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538420 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538425 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538429 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538433 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538438 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538442 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538446 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538450 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538455 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538459 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 14:59:03.545435 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538463 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538467 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538471 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538475 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538480 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538484 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538489 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538493 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538497 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538502 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538506 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538510 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538514 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538518 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538522 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538526 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538531 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538535 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538539 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538544 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 14:59:03.545977 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538548 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538552 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538557 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538562 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538566 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538570 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:03.538574 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.538582 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.538752 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.541354 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.542892 2562 server.go:1019] "Starting client certificate rotation" Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.543368 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.544092 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 14:59:03.546458 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.545981 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 14:59:03.550792 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.548927 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 14:59:03.558819 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.558793 2562 log.go:25] "Validated CRI v1 runtime API" Jun 11 14:59:03.563262 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.563248 2562 log.go:25] "Validated CRI v1 image API" Jun 11 14:59:03.563969 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.563954 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 14:59:03.566356 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.566333 2562 fs.go:135] Filesystem UUIDs: map[2f5e0105-eae4-4fb3-b238-41e71d752e4e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f57c1183-34f4-48e9-9e22-4a6149391cda:/dev/nvme0n1p3] Jun 11 14:59:03.566404 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.566356 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 14:59:03.571416 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.571315 2562 manager.go:216] Machine: {Timestamp:2026-06-11 14:59:03.569967733 +0000 UTC m=+0.346599444 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099576 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e2e75ab7d87a65a5b153cf29aded1 SystemUUID:ec2e2e75-ab7d-87a6-5a5b-153cf29aded1 BootID:5b00e365-a93c-4a58-9fd9-380dd7fbdcac Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d2:cb:92:65:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d2:cb:92:65:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:07:d1:0a:28:74 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 14:59:03.571416 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.571410 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 14:59:03.571532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.571522 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 14:59:03.572353 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.572338 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 14:59:03.574641 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.574618 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 14:59:03.574798 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.574644 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-77.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 14:59:03.575452 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.575442 2562 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 14:59:03.575483 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.575454 2562 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 14:59:03.575483 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.575462 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 14:59:03.576918 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.576907 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 14:59:03.577771 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.577761 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 14:59:03.580643 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.580631 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 11 14:59:03.580764 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.580754 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 14:59:03.580802 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.580783 2562 kubelet.go:486] "Attempting to sync node with API server" Jun 11 14:59:03.580802 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.580793 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 14:59:03.580853 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.580806 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 14:59:03.580853 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.580815 2562 kubelet.go:398] "Adding apiserver pod source" Jun 11 14:59:03.580853 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.580823 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 14:59:03.581250 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.581236 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 14:59:03.581302 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.581263 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 14:59:03.583756 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.583738 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 14:59:03.585151 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.585105 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 14:59:03.585193 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.585154 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 14:59:03.586415 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586404 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 14:59:03.586448 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586420 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 14:59:03.586448 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586427 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 14:59:03.586448 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586432 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 14:59:03.586448 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586441 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 14:59:03.586448 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586448 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 14:59:03.586578 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586457 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 14:59:03.586578 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586463 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 14:59:03.586578 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586470 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 14:59:03.586578 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586475 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 14:59:03.586578 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586491 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 14:59:03.586578 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.586497 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 14:59:03.587257 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.587246 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 14:59:03.587292 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.587259 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 14:59:03.590595 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.590581 2562 server.go:1268] "Started kubelet" Jun 11 14:59:03.591336 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.591264 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 14:59:03.591849 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.591656 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 14:59:03.591849 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.591068 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 14:59:03.592033 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.591902 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 14:59:03.591847 ip-10-0-140-77 systemd[1]: Started Kubernetes Kubelet. Jun 11 14:59:03.592551 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.592050 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-77.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 14:59:03.593133 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.591899 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 14:59:03.593735 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.593685 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 14:59:03.595209 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.595190 2562 server.go:310] "Adding debug handlers to kubelet server" Jun 11 14:59:03.599532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.599479 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 14:59:03.599532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.599485 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 14:59:03.599660 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.599594 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:03.600169 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600152 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 14:59:03.600340 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600314 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 14:59:03.600340 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600335 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 14:59:03.600478 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600407 2562 factory.go:55] Registering systemd factory Jun 11 14:59:03.600478 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600431 2562 reconstruct.go:97] "Volume reconstruction finished" Jun 11 14:59:03.600478 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600440 2562 reconciler.go:29] "Reconciler: start to sync state" Jun 11 14:59:03.600478 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600445 2562 factory.go:223] Registration of the systemd container factory successfully Jun 11 14:59:03.600641 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.599421 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-77.ec2.internal.18b80ed10936946d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-77.ec2.internal,UID:ip-10-0-140-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-77.ec2.internal,},FirstTimestamp:2026-06-11 14:59:03.590556781 +0000 UTC m=+0.367188493,LastTimestamp:2026-06-11 14:59:03.590556781 +0000 UTC m=+0.367188493,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-77.ec2.internal,}" Jun 11 14:59:03.600641 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.600535 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 11 14:59:03.600768 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600665 2562 factory.go:153] Registering CRI-O factory Jun 11 14:59:03.600768 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600680 2562 factory.go:223] Registration of the crio container factory successfully Jun 11 14:59:03.600768 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600757 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 14:59:03.600904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600781 2562 factory.go:103] Registering Raw factory Jun 11 14:59:03.600904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.600804 2562 manager.go:1196] Started watching for new ooms in manager Jun 11 14:59:03.601147 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.601134 2562 manager.go:349] Starting recovery of all containers Jun 11 14:59:03.610556 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.610519 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 11 14:59:03.610556 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.610510 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 11 14:59:03.610954 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.610937 2562 manager.go:354] Recovery completed Jun 11 14:59:03.612937 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.612921 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kmdf" Jun 11 14:59:03.616257 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.616245 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 14:59:03.618064 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.618049 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientMemory" Jun 11 14:59:03.618127 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.618075 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 14:59:03.618127 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.618085 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientPID" Jun 11 14:59:03.618594 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.618580 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 14:59:03.618594 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.618593 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 14:59:03.618690 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.618601 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 14:59:03.618690 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.618607 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 11 14:59:03.619420 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.619403 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientMemory" Jun 11 14:59:03.619497 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.619432 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 14:59:03.619497 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.619443 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientPID" Jun 11 14:59:03.619889 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.619876 2562 policy_none.go:49] "None policy: Start" Jun 11 14:59:03.619933 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.619896 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 14:59:03.619970 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.619911 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 14:59:03.620336 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.620321 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4kmdf" Jun 11 14:59:03.621018 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.621005 2562 policy_none.go:47] "Start" Jun 11 14:59:03.646962 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.646942 2562 manager.go:337] "Starting Device Plugin manager" Jun 11 14:59:03.647049 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.646998 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 14:59:03.647049 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.647008 2562 server.go:87] "Starting device plugin registration server" Jun 11 14:59:03.647283 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.647269 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 14:59:03.647381 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.647283 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 14:59:03.647381 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.647371 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 14:59:03.647525 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.647493 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 14:59:03.647525 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.647508 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 14:59:03.648051 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.648027 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 14:59:03.648142 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.648080 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:03.667697 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.667667 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 14:59:03.674739 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.668785 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 14:59:03.674739 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.668816 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 14:59:03.674739 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.668836 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 14:59:03.674739 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.668918 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 11 14:59:03.674739 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.671244 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 14:59:03.747574 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.747551 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 14:59:03.748492 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.748471 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientMemory" Jun 11 14:59:03.748588 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.748510 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 14:59:03.748588 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.748524 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientPID" Jun 11 14:59:03.748588 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.748549 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.757131 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.757115 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.757196 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.757137 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-77.ec2.internal\": node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:03.769474 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.769413 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal"] Jun 11 14:59:03.769549 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.769476 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 14:59:03.769587 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.769569 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:03.770844 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.770829 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientMemory" Jun 11 14:59:03.770925 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.770854 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 14:59:03.770925 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.770864 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientPID" Jun 11 14:59:03.772057 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.772046 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 14:59:03.772174 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.772159 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.772210 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.772187 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 14:59:03.772684 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.772670 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientMemory" Jun 11 14:59:03.772768 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.772693 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 14:59:03.772768 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.772704 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientPID" Jun 11 14:59:03.773230 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.773216 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientMemory" Jun 11 14:59:03.773286 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.773241 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 14:59:03.773286 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.773252 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientPID" Jun 11 14:59:03.774272 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.774259 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.774321 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.774285 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 14:59:03.774984 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.774969 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientMemory" Jun 11 14:59:03.775060 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.774995 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 14:59:03.775060 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.775005 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeHasSufficientPID" Jun 11 14:59:03.801450 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.801425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d21059ae7ce928c107437710156b4ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal\" (UID: \"5d21059ae7ce928c107437710156b4ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.801560 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.801455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21059ae7ce928c107437710156b4ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal\" (UID: \"5d21059ae7ce928c107437710156b4ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.801560 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.801482 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d985ea8e747104fff16cbaa5995c0307-config\") pod \"kube-apiserver-proxy-ip-10-0-140-77.ec2.internal\" (UID: \"d985ea8e747104fff16cbaa5995c0307\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.801560 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.801491 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-77.ec2.internal\" not found" node="ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.806003 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.805987 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-77.ec2.internal\" not found" node="ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.869980 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.869948 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:03.902100 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.902068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d21059ae7ce928c107437710156b4ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal\" (UID: \"5d21059ae7ce928c107437710156b4ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.902184 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.902104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21059ae7ce928c107437710156b4ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal\" (UID: \"5d21059ae7ce928c107437710156b4ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.902184 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.902121 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d985ea8e747104fff16cbaa5995c0307-config\") pod \"kube-apiserver-proxy-ip-10-0-140-77.ec2.internal\" (UID: \"d985ea8e747104fff16cbaa5995c0307\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.902184 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.902172 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d21059ae7ce928c107437710156b4ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal\" (UID: \"5d21059ae7ce928c107437710156b4ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.902289 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.902250 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d985ea8e747104fff16cbaa5995c0307-config\") pod \"kube-apiserver-proxy-ip-10-0-140-77.ec2.internal\" (UID: \"d985ea8e747104fff16cbaa5995c0307\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.902323 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:03.902289 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d21059ae7ce928c107437710156b4ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal\" (UID: \"5d21059ae7ce928c107437710156b4ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:03.970266 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:03.970231 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:04.070870 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:04.070786 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:04.105504 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.105467 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:04.110298 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.110284 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" Jun 11 14:59:04.171055 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:04.171020 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:04.271560 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:04.271532 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:04.372099 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:04.372066 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:04.472572 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:04.472536 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:04.477293 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.477277 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 14:59:04.542549 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.542517 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 14:59:04.543156 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.542649 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 14:59:04.543156 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.542691 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 14:59:04.572782 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:04.572751 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:04.622701 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.622642 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 14:54:03 +0000 UTC" deadline="2028-01-05 05:32:54.74973696 +0000 UTC" Jun 11 14:59:04.622701 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.622674 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13742h33m50.127069033s" Jun 11 14:59:04.673020 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:04.672855 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-77.ec2.internal\" not found" Jun 11 14:59:04.676622 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.676602 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 14:59:04.699921 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.699896 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" Jun 11 14:59:04.710357 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.710328 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 14:59:04.712161 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.712142 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" Jun 11 14:59:04.718948 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.718930 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 14:59:04.723575 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:04.723532 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d21059ae7ce928c107437710156b4ba.slice/crio-fe925cb6ccedd1ae77e40e67a47f2ba31774e239fec4eb37080c307382d20efc WatchSource:0}: Error finding container fe925cb6ccedd1ae77e40e67a47f2ba31774e239fec4eb37080c307382d20efc: Status 404 returned error can't find the container with id fe925cb6ccedd1ae77e40e67a47f2ba31774e239fec4eb37080c307382d20efc Jun 11 14:59:04.724045 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:04.724018 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd985ea8e747104fff16cbaa5995c0307.slice/crio-b62ba0ec8b579e865e0003e3578a3c39394608ffcf3f13257082a2835e8b38d5 WatchSource:0}: Error finding container b62ba0ec8b579e865e0003e3578a3c39394608ffcf3f13257082a2835e8b38d5: Status 404 returned error can't find the container with id b62ba0ec8b579e865e0003e3578a3c39394608ffcf3f13257082a2835e8b38d5 Jun 11 14:59:04.728564 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.728549 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 14:59:04.975007 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:04.974921 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 14:59:05.393194 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.393010 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 14:59:05.582314 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.582279 2562 apiserver.go:52] "Watching apiserver" Jun 11 14:59:05.590519 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.590492 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 14:59:05.593303 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.593265 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q","openshift-dns/node-resolver-bjhzp","openshift-image-registry/node-ca-99vwj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal","openshift-multus/multus-additional-cni-plugins-529bv","openshift-multus/network-metrics-daemon-jxjxg","kube-system/konnectivity-agent-8758w","openshift-cluster-node-tuning-operator/tuned-thlvs","openshift-multus/multus-fhks6","openshift-network-diagnostics/network-check-target-5277h","openshift-network-operator/iptables-alerter-bkpmb","openshift-ovn-kubernetes/ovnkube-node-h9k9f","kube-system/global-pull-secret-syncer-2zl8k"] Jun 11 14:59:05.595029 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.595008 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:05.595122 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.595098 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:05.596038 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.596018 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:05.596144 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.596090 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:05.597155 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.597136 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.598266 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.598213 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.599462 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.599442 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.599573 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.599528 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:05.599900 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.599866 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2lwp8\"" Jun 11 14:59:05.600400 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.600364 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 14:59:05.600665 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.600647 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.602090 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.602072 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.604133 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.603982 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.605735 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.605703 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.607577 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.607237 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.608453 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.608431 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:05.610232 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.610214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.612766 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ll62\" (UniqueName: \"kubernetes.io/projected/46e97f49-b264-4a5e-8f66-9ae0ea812103-kube-api-access-6ll62\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:05.612856 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6f5d097-246f-4602-a126-182e5018d288-hosts-file\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.612856 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b6572eff-fb22-4baa-b130-995f9284a608-dbus\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.612955 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612856 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-system-cni-dir\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.612955 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfrtc\" (UniqueName: \"kubernetes.io/projected/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-kube-api-access-rfrtc\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.612955 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:05.612955 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612948 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f5d097-246f-4602-a126-182e5018d288-tmp-dir\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.613143 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b6572eff-fb22-4baa-b130-995f9284a608-kubelet-config\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.613143 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.612999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.613143 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cnibin\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.613143 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-os-release\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.613143 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613071 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.613143 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgmtt\" (UniqueName: \"kubernetes.io/projected/d6f5d097-246f-4602-a126-182e5018d288-kube-api-access-mgmtt\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.613417 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c998f7d-b189-4843-a760-069af193cde6-host\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.613417 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613227 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.613417 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613267 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.613417 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613319 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.613417 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:05.613417 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8c998f7d-b189-4843-a760-069af193cde6-serviceca\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.613693 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.613419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl66z\" (UniqueName: \"kubernetes.io/projected/8c998f7d-b189-4843-a760-069af193cde6-kube-api-access-cl66z\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.614775 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.614529 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 14:59:05.617167 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.617144 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 14:59:05.617493 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.617474 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 14:59:05.617560 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.617499 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 14:59:05.617820 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.617796 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 14:59:05.618301 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.618285 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 14:59:05.618593 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.618566 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 14:59:05.618924 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.618908 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 14:59:05.619003 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.618923 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 14:59:05.619269 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.619252 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 14:59:05.619635 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.619619 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 14:59:05.620081 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620065 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 14:59:05.620136 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620096 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 14:59:05.620356 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620338 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 14:59:05.620444 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620427 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 14:59:05.620530 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620517 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 14:59:05.620597 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620581 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 14:59:05.620861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620846 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 14:59:05.620904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620858 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 14:59:05.621123 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.621110 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pnqn6\"" Jun 11 14:59:05.621382 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.621362 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k7kc8\"" Jun 11 14:59:05.621494 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.620852 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 14:59:05.621985 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.621956 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m7d94\"" Jun 11 14:59:05.622081 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.621985 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pdx72\"" Jun 11 14:59:05.622081 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.622013 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 14:59:05.622194 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.622174 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 14:59:05.623251 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623234 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-svq54\"" Jun 11 14:59:05.623630 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623277 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 14:59:05.623780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623291 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 14:59:05.623851 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623442 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2fkt4\"" Jun 11 14:59:05.623911 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623481 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 14:59:05.623966 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623506 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 14:59:05.623966 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623523 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 14:59:05.624057 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623534 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5wf6k\"" Jun 11 14:59:05.624057 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.623577 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-l5bfg\"" Jun 11 14:59:05.631011 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.630990 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 14:59:05.649488 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.649466 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kl789" Jun 11 14:59:05.656060 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.656037 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kl789" Jun 11 14:59:05.675004 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.674960 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" event={"ID":"5d21059ae7ce928c107437710156b4ba","Type":"ContainerStarted","Data":"fe925cb6ccedd1ae77e40e67a47f2ba31774e239fec4eb37080c307382d20efc"} Jun 11 14:59:05.676147 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.676118 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" event={"ID":"d985ea8e747104fff16cbaa5995c0307","Type":"ContainerStarted","Data":"b62ba0ec8b579e865e0003e3578a3c39394608ffcf3f13257082a2835e8b38d5"} Jun 11 14:59:05.701081 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.701056 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 14:59:05.714309 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6f5d097-246f-4602-a126-182e5018d288-hosts-file\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.714473 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b06bf046-e052-4cfc-902b-05f689ad740d-etc-tuned\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.714473 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b06bf046-e052-4cfc-902b-05f689ad740d-tmp\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.714473 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-ovnkube-config\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.714473 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714417 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d6f5d097-246f-4602-a126-182e5018d288-hosts-file\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.714473 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-socket-dir-parent\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.714473 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714470 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/151bfb0e-d6da-4843-8f17-b1a14999fd3c-agent-certs\") pod \"konnectivity-agent-8758w\" (UID: \"151bfb0e-d6da-4843-8f17-b1a14999fd3c\") " pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/095e3504-ec5b-48dd-9683-9d235ff3bbcc-iptables-alerter-script\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b6572eff-fb22-4baa-b130-995f9284a608-kubelet-config\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cnibin\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-run\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b6572eff-fb22-4baa-b130-995f9284a608-kubelet-config\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-kubelet\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714613 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-node-log\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-device-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714644 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cnibin\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgmtt\" (UniqueName: \"kubernetes.io/projected/d6f5d097-246f-4602-a126-182e5018d288-kube-api-access-mgmtt\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.714780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-kubernetes\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714802 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2k74\" (UniqueName: \"kubernetes.io/projected/b06bf046-e052-4cfc-902b-05f689ad740d-kube-api-access-c2k74\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714844 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-cni-bin\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714867 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-cni-netd\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-ovnkube-script-lib\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714915 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m8wf\" (UniqueName: \"kubernetes.io/projected/f798fa81-f718-4b9c-84e8-568b640625de-kube-api-access-2m8wf\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl66z\" (UniqueName: \"kubernetes.io/projected/8c998f7d-b189-4843-a760-069af193cde6-kube-api-access-cl66z\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-systemd-units\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.714994 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-run-netns\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-env-overrides\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-sys-fs\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715080 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-cni-multus\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-hostroot\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715153 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-conf-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-log-socket\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715247 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysconfig\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.715369 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ll62\" (UniqueName: \"kubernetes.io/projected/46e97f49-b264-4a5e-8f66-9ae0ea812103-kube-api-access-6ll62\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfrtc\" (UniqueName: \"kubernetes.io/projected/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-kube-api-access-rfrtc\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-modprobe-d\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-var-lib-kubelet\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f5d097-246f-4602-a126-182e5018d288-tmp-dir\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715415 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-sys\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715525 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-host\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7jt5\" (UniqueName: \"kubernetes.io/projected/96157a6e-8a7c-444c-807e-8967dc056298-kube-api-access-b7jt5\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-cni-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715670 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96157a6e-8a7c-444c-807e-8967dc056298-cni-binary-copy\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-k8s-cni-cncf-io\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715731 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8c998f7d-b189-4843-a760-069af193cde6-serviceca\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/945a32d3-e116-4317-b5c7-23856c9645fb-ovn-node-metrics-cert\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.715968 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-socket-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/151bfb0e-d6da-4843-8f17-b1a14999fd3c-konnectivity-ca\") pod \"konnectivity-agent-8758w\" (UID: \"151bfb0e-d6da-4843-8f17-b1a14999fd3c\") " pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715836 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysctl-conf\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-systemd\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715879 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-etc-selinux\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-cnibin\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-etc-kubernetes\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.715976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-os-release\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716002 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-systemd\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-lib-modules\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-system-cni-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716099 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysctl-d\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf762\" (UniqueName: \"kubernetes.io/projected/095e3504-ec5b-48dd-9683-9d235ff3bbcc-kube-api-access-lf762\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-slash\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716229 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-netns\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.716582 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-kubelet\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716282 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-run-ovn-kubernetes\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b6572eff-fb22-4baa-b130-995f9284a608-dbus\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716343 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-system-cni-dir\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716415 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-os-release\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-cni-bin\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716500 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8c998f7d-b189-4843-a760-069af193cde6-serviceca\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-registration-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/095e3504-ec5b-48dd-9683-9d235ff3bbcc-host-slash\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716584 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-os-release\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.716469 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.716767 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret podName:b6572eff-fb22-4baa-b130-995f9284a608 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:06.216745037 +0000 UTC m=+2.993376756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret") pod "global-pull-secret-syncer-2zl8k" (UID: "b6572eff-fb22-4baa-b130-995f9284a608") : object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716793 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.716868 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:05.717327 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c998f7d-b189-4843-a760-069af193cde6-host\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716885 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f5d097-246f-4602-a126-182e5018d288-tmp-dir\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.716921 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs podName:46e97f49-b264-4a5e-8f66-9ae0ea812103 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:06.216905711 +0000 UTC m=+2.993537420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs") pod "network-metrics-daemon-jxjxg" (UID: "46e97f49-b264-4a5e-8f66-9ae0ea812103") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716935 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b6572eff-fb22-4baa-b130-995f9284a608-dbus\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-var-lib-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716968 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c998f7d-b189-4843-a760-069af193cde6-host\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.716972 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-system-cni-dir\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.717006 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-etc-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.717039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96157a6e-8a7c-444c-807e-8967dc056298-multus-daemon-config\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.717068 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.717077 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-ovn\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.717107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.717135 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdt6p\" (UniqueName: \"kubernetes.io/projected/945a32d3-e116-4317-b5c7-23856c9645fb-kube-api-access-sdt6p\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.718019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.717174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-multus-certs\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.721374 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.721315 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 14:59:05.722611 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.722561 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 14:59:05.722611 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.722586 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 14:59:05.722795 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.722617 2562 projected.go:196] Error preparing data for projected volume kube-api-access-c45rd for pod openshift-network-diagnostics/network-check-target-5277h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:05.722795 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:05.722703 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd podName:159b39a6-357d-4362-800e-2bb1d2fca812 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:06.222688511 +0000 UTC m=+2.999320227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c45rd" (UniqueName: "kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd") pod "network-check-target-5277h" (UID: "159b39a6-357d-4362-800e-2bb1d2fca812") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:05.725133 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.725115 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl66z\" (UniqueName: \"kubernetes.io/projected/8c998f7d-b189-4843-a760-069af193cde6-kube-api-access-cl66z\") pod \"node-ca-99vwj\" (UID: \"8c998f7d-b189-4843-a760-069af193cde6\") " pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.725211 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.725152 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgmtt\" (UniqueName: \"kubernetes.io/projected/d6f5d097-246f-4602-a126-182e5018d288-kube-api-access-mgmtt\") pod \"node-resolver-bjhzp\" (UID: \"d6f5d097-246f-4602-a126-182e5018d288\") " pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.725656 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.725636 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfrtc\" (UniqueName: \"kubernetes.io/projected/b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7-kube-api-access-rfrtc\") pod \"multus-additional-cni-plugins-529bv\" (UID: \"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7\") " pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.725913 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.725885 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ll62\" (UniqueName: \"kubernetes.io/projected/46e97f49-b264-4a5e-8f66-9ae0ea812103-kube-api-access-6ll62\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:05.818489 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-run-ovn-kubernetes\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.818489 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-os-release\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.818710 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-cni-bin\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.818710 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.818710 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818571 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-run-ovn-kubernetes\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.818710 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818588 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.818710 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818606 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-cni-bin\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.818710 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818682 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-os-release\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.818710 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-registration-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818748 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/095e3504-ec5b-48dd-9683-9d235ff3bbcc-host-slash\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-var-lib-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-etc-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818845 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-var-lib-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818848 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/095e3504-ec5b-48dd-9683-9d235ff3bbcc-host-slash\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818853 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-registration-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818898 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-etc-openvswitch\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818924 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96157a6e-8a7c-444c-807e-8967dc056298-multus-daemon-config\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-ovn\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818955 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdt6p\" (UniqueName: \"kubernetes.io/projected/945a32d3-e116-4317-b5c7-23856c9645fb-kube-api-access-sdt6p\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.818999 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-multus-certs\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.819030 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b06bf046-e052-4cfc-902b-05f689ad740d-etc-tuned\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819045 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b06bf046-e052-4cfc-902b-05f689ad740d-tmp\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819069 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-ovnkube-config\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-socket-dir-parent\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/151bfb0e-d6da-4843-8f17-b1a14999fd3c-agent-certs\") pod \"konnectivity-agent-8758w\" (UID: \"151bfb0e-d6da-4843-8f17-b1a14999fd3c\") " pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819114 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/095e3504-ec5b-48dd-9683-9d235ff3bbcc-iptables-alerter-script\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-run\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-kubelet\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-node-log\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-device-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-kubernetes\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2k74\" (UniqueName: \"kubernetes.io/projected/b06bf046-e052-4cfc-902b-05f689ad740d-kube-api-access-c2k74\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-cni-bin\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-cni-netd\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819313 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-ovnkube-script-lib\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m8wf\" (UniqueName: \"kubernetes.io/projected/f798fa81-f718-4b9c-84e8-568b640625de-kube-api-access-2m8wf\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.819632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-systemd-units\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-run-netns\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-env-overrides\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-sys-fs\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819446 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-kubelet\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819453 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-cni-bin\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-sys-fs\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819504 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96157a6e-8a7c-444c-807e-8967dc056298-multus-daemon-config\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819531 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819524 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-ovn\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819507 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-cni-netd\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819582 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-device-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819591 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819602 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-multus-certs\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819649 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-run-netns\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819696 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-systemd-units\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.819766 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-kubernetes\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.820548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-cni-multus\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820209 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-ovnkube-script-lib\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820240 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-node-log\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-hostroot\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820281 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-cni-multus\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820300 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-hostroot\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-conf-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820198 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-env-overrides\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820339 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-run\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820359 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-conf-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820358 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-socket-dir-parent\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-log-socket\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820408 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysconfig\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-modprobe-d\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820461 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-var-lib-kubelet\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-sys\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-host\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820539 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/095e3504-ec5b-48dd-9683-9d235ff3bbcc-iptables-alerter-script\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.821440 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7jt5\" (UniqueName: \"kubernetes.io/projected/96157a6e-8a7c-444c-807e-8967dc056298-kube-api-access-b7jt5\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-cni-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96157a6e-8a7c-444c-807e-8967dc056298-cni-binary-copy\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820656 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-k8s-cni-cncf-io\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/945a32d3-e116-4317-b5c7-23856c9645fb-ovn-node-metrics-cert\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-socket-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/151bfb0e-d6da-4843-8f17-b1a14999fd3c-konnectivity-ca\") pod \"konnectivity-agent-8758w\" (UID: \"151bfb0e-d6da-4843-8f17-b1a14999fd3c\") " pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysctl-conf\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820804 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-var-lib-kubelet\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-modprobe-d\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820867 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/945a32d3-e116-4317-b5c7-23856c9645fb-ovnkube-config\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820874 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-sys\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-systemd\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820924 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-host\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820924 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysconfig\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-etc-selinux\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820954 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-log-socket\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.820984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-cnibin\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.822535 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-etc-kubernetes\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-systemd\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-socket-dir\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821083 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysctl-conf\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821105 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-run-systemd\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821161 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-lib-modules\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-lib-modules\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821175 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f798fa81-f718-4b9c-84e8-568b640625de-etc-selinux\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-system-cni-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821216 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-cnibin\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysctl-d\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821276 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-system-cni-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf762\" (UniqueName: \"kubernetes.io/projected/095e3504-ec5b-48dd-9683-9d235ff3bbcc-kube-api-access-lf762\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821316 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-etc-kubernetes\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-slash\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-netns\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-kubelet\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821489 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-var-lib-kubelet\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.823537 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821537 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-systemd\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821572 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/945a32d3-e116-4317-b5c7-23856c9645fb-host-slash\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821613 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b06bf046-e052-4cfc-902b-05f689ad740d-etc-sysctl-d\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821609 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-netns\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821667 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-host-run-k8s-cni-cncf-io\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821681 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96157a6e-8a7c-444c-807e-8967dc056298-cni-binary-copy\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.821684 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96157a6e-8a7c-444c-807e-8967dc056298-multus-cni-dir\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.822271 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/151bfb0e-d6da-4843-8f17-b1a14999fd3c-konnectivity-ca\") pod \"konnectivity-agent-8758w\" (UID: \"151bfb0e-d6da-4843-8f17-b1a14999fd3c\") " pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.823049 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b06bf046-e052-4cfc-902b-05f689ad740d-etc-tuned\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.823248 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b06bf046-e052-4cfc-902b-05f689ad740d-tmp\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.823898 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/945a32d3-e116-4317-b5c7-23856c9645fb-ovn-node-metrics-cert\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.824396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.824029 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/151bfb0e-d6da-4843-8f17-b1a14999fd3c-agent-certs\") pod \"konnectivity-agent-8758w\" (UID: \"151bfb0e-d6da-4843-8f17-b1a14999fd3c\") " pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:05.830705 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.830674 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m8wf\" (UniqueName: \"kubernetes.io/projected/f798fa81-f718-4b9c-84e8-568b640625de-kube-api-access-2m8wf\") pod \"aws-ebs-csi-driver-node-z6d6q\" (UID: \"f798fa81-f718-4b9c-84e8-568b640625de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.831464 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.831441 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdt6p\" (UniqueName: \"kubernetes.io/projected/945a32d3-e116-4317-b5c7-23856c9645fb-kube-api-access-sdt6p\") pod \"ovnkube-node-h9k9f\" (UID: \"945a32d3-e116-4317-b5c7-23856c9645fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.831701 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.831679 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2k74\" (UniqueName: \"kubernetes.io/projected/b06bf046-e052-4cfc-902b-05f689ad740d-kube-api-access-c2k74\") pod \"tuned-thlvs\" (UID: \"b06bf046-e052-4cfc-902b-05f689ad740d\") " pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.831919 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.831898 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf762\" (UniqueName: \"kubernetes.io/projected/095e3504-ec5b-48dd-9683-9d235ff3bbcc-kube-api-access-lf762\") pod \"iptables-alerter-bkpmb\" (UID: \"095e3504-ec5b-48dd-9683-9d235ff3bbcc\") " pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:05.832375 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.832354 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7jt5\" (UniqueName: \"kubernetes.io/projected/96157a6e-8a7c-444c-807e-8967dc056298-kube-api-access-b7jt5\") pod \"multus-fhks6\" (UID: \"96157a6e-8a7c-444c-807e-8967dc056298\") " pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.911371 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.911290 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bjhzp" Jun 11 14:59:05.920211 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.920190 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-99vwj" Jun 11 14:59:05.930946 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.930924 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-529bv" Jun 11 14:59:05.937818 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.937800 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-thlvs" Jun 11 14:59:05.945417 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.945395 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:05.952495 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.952473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" Jun 11 14:59:05.959503 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.959473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fhks6" Jun 11 14:59:05.967395 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.967372 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:05.974385 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:05.974367 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bkpmb" Jun 11 14:59:06.223707 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.223628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:06.223707 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.223683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.223729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:06.223808 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:06.223816 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:06.223827 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:06.223846 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:06.223860 2562 projected.go:196] Error preparing data for projected volume kube-api-access-c45rd for pod openshift-network-diagnostics/network-check-target-5277h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:06.223875 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs podName:46e97f49-b264-4a5e-8f66-9ae0ea812103 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:07.223859512 +0000 UTC m=+4.000491211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs") pod "network-metrics-daemon-jxjxg" (UID: "46e97f49-b264-4a5e-8f66-9ae0ea812103") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:06.223889 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret podName:b6572eff-fb22-4baa-b130-995f9284a608 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:07.223883229 +0000 UTC m=+4.000514928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret") pod "global-pull-secret-syncer-2zl8k" (UID: "b6572eff-fb22-4baa-b130-995f9284a608") : object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:06.223910 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:06.223913 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd podName:159b39a6-357d-4362-800e-2bb1d2fca812 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:07.223895716 +0000 UTC m=+4.000527430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45rd" (UniqueName: "kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd") pod "network-check-target-5277h" (UID: "159b39a6-357d-4362-800e-2bb1d2fca812") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:06.457991 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.457962 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 14:59:06.643523 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.643477 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96157a6e_8a7c_444c_807e_8967dc056298.slice/crio-edef3ef6e4833cb6ad864f7fea132f111d39446f99abdd153133a909d71c3490 WatchSource:0}: Error finding container edef3ef6e4833cb6ad864f7fea132f111d39446f99abdd153133a909d71c3490: Status 404 returned error can't find the container with id edef3ef6e4833cb6ad864f7fea132f111d39446f99abdd153133a909d71c3490 Jun 11 14:59:06.643929 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.643840 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf798fa81_f718_4b9c_84e8_568b640625de.slice/crio-a53ae83b925077d95631b2fc717273639bb112e5bf3b7ba6fbeafa0d68e366a0 WatchSource:0}: Error finding container a53ae83b925077d95631b2fc717273639bb112e5bf3b7ba6fbeafa0d68e366a0: Status 404 returned error can't find the container with id a53ae83b925077d95631b2fc717273639bb112e5bf3b7ba6fbeafa0d68e366a0 Jun 11 14:59:06.649185 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.649160 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c998f7d_b189_4843_a760_069af193cde6.slice/crio-80b1f1a47fa8acd1c4a6d1b84acee9b375ddd17fa585870be0a47147e2eeca01 WatchSource:0}: Error finding container 80b1f1a47fa8acd1c4a6d1b84acee9b375ddd17fa585870be0a47147e2eeca01: Status 404 returned error can't find the container with id 80b1f1a47fa8acd1c4a6d1b84acee9b375ddd17fa585870be0a47147e2eeca01 Jun 11 14:59:06.650205 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.650185 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f5d097_246f_4602_a126_182e5018d288.slice/crio-645840b10a6035e1ff4f2328f0cffc941ba07e2291e816886aec016c4b80b8c5 WatchSource:0}: Error finding container 645840b10a6035e1ff4f2328f0cffc941ba07e2291e816886aec016c4b80b8c5: Status 404 returned error can't find the container with id 645840b10a6035e1ff4f2328f0cffc941ba07e2291e816886aec016c4b80b8c5 Jun 11 14:59:06.650753 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.650732 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02b7dc5_d226_4d1a_97f4_f2c6e4d894d7.slice/crio-cfc5cc7b57b01ed328b395d3ccab03bc70e0a90862935a393da240e829c6233d WatchSource:0}: Error finding container cfc5cc7b57b01ed328b395d3ccab03bc70e0a90862935a393da240e829c6233d: Status 404 returned error can't find the container with id cfc5cc7b57b01ed328b395d3ccab03bc70e0a90862935a393da240e829c6233d Jun 11 14:59:06.652947 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.652898 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb06bf046_e052_4cfc_902b_05f689ad740d.slice/crio-0e1e9c0bdf544d7f66e32816b5868900bab3fe1c194a6e8a4e92c491f4d3c0f6 WatchSource:0}: Error finding container 0e1e9c0bdf544d7f66e32816b5868900bab3fe1c194a6e8a4e92c491f4d3c0f6: Status 404 returned error can't find the container with id 0e1e9c0bdf544d7f66e32816b5868900bab3fe1c194a6e8a4e92c491f4d3c0f6 Jun 11 14:59:06.654378 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.654358 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151bfb0e_d6da_4843_8f17_b1a14999fd3c.slice/crio-6dec8387b746f9e2529532f414cc9be67554710679c194821964d5db07a74ab0 WatchSource:0}: Error finding container 6dec8387b746f9e2529532f414cc9be67554710679c194821964d5db07a74ab0: Status 404 returned error can't find the container with id 6dec8387b746f9e2529532f414cc9be67554710679c194821964d5db07a74ab0 Jun 11 14:59:06.655430 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.655380 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945a32d3_e116_4317_b5c7_23856c9645fb.slice/crio-fe8e1ca6644b416c807eb4c37a8f5e4dacd88349fb415597c2fbbd941b19365f WatchSource:0}: Error finding container fe8e1ca6644b416c807eb4c37a8f5e4dacd88349fb415597c2fbbd941b19365f: Status 404 returned error can't find the container with id fe8e1ca6644b416c807eb4c37a8f5e4dacd88349fb415597c2fbbd941b19365f Jun 11 14:59:06.658124 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:06.657041 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095e3504_ec5b_48dd_9683_9d235ff3bbcc.slice/crio-35d8266243720bf73038aa150374b6f300a73f752d7e62a8a1a8cd1d1751065f WatchSource:0}: Error finding container 35d8266243720bf73038aa150374b6f300a73f752d7e62a8a1a8cd1d1751065f: Status 404 returned error can't find the container with id 35d8266243720bf73038aa150374b6f300a73f752d7e62a8a1a8cd1d1751065f Jun 11 14:59:06.659633 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.659420 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 14:54:05 +0000 UTC" deadline="2028-01-24 13:18:27.695322634 +0000 UTC" Jun 11 14:59:06.659633 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.659457 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14206h19m21.035868522s" Jun 11 14:59:06.682177 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.682152 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" event={"ID":"f798fa81-f718-4b9c-84e8-568b640625de","Type":"ContainerStarted","Data":"a53ae83b925077d95631b2fc717273639bb112e5bf3b7ba6fbeafa0d68e366a0"} Jun 11 14:59:06.684970 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.684942 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fhks6" event={"ID":"96157a6e-8a7c-444c-807e-8967dc056298","Type":"ContainerStarted","Data":"edef3ef6e4833cb6ad864f7fea132f111d39446f99abdd153133a909d71c3490"} Jun 11 14:59:06.686133 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.686108 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bkpmb" event={"ID":"095e3504-ec5b-48dd-9683-9d235ff3bbcc","Type":"ContainerStarted","Data":"35d8266243720bf73038aa150374b6f300a73f752d7e62a8a1a8cd1d1751065f"} Jun 11 14:59:06.687503 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.687476 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bjhzp" event={"ID":"d6f5d097-246f-4602-a126-182e5018d288","Type":"ContainerStarted","Data":"645840b10a6035e1ff4f2328f0cffc941ba07e2291e816886aec016c4b80b8c5"} Jun 11 14:59:06.688685 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.688631 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-99vwj" event={"ID":"8c998f7d-b189-4843-a760-069af193cde6","Type":"ContainerStarted","Data":"80b1f1a47fa8acd1c4a6d1b84acee9b375ddd17fa585870be0a47147e2eeca01"} Jun 11 14:59:06.689955 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.689934 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"fe8e1ca6644b416c807eb4c37a8f5e4dacd88349fb415597c2fbbd941b19365f"} Jun 11 14:59:06.690953 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.690933 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-thlvs" event={"ID":"b06bf046-e052-4cfc-902b-05f689ad740d","Type":"ContainerStarted","Data":"0e1e9c0bdf544d7f66e32816b5868900bab3fe1c194a6e8a4e92c491f4d3c0f6"} Jun 11 14:59:06.691883 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.691865 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8758w" event={"ID":"151bfb0e-d6da-4843-8f17-b1a14999fd3c","Type":"ContainerStarted","Data":"6dec8387b746f9e2529532f414cc9be67554710679c194821964d5db07a74ab0"} Jun 11 14:59:06.692808 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:06.692790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-529bv" event={"ID":"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7","Type":"ContainerStarted","Data":"cfc5cc7b57b01ed328b395d3ccab03bc70e0a90862935a393da240e829c6233d"} Jun 11 14:59:07.232258 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.232173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:07.232258 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.232238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:07.232485 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.232268 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:07.232485 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.232398 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:07.232485 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.232448 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 14:59:07.232485 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.232466 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret podName:b6572eff-fb22-4baa-b130-995f9284a608 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:09.232446316 +0000 UTC m=+6.009078016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret") pod "global-pull-secret-syncer-2zl8k" (UID: "b6572eff-fb22-4baa-b130-995f9284a608") : object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:07.232485 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.232473 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 14:59:07.232485 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.232398 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:07.232485 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.232487 2562 projected.go:196] Error preparing data for projected volume kube-api-access-c45rd for pod openshift-network-diagnostics/network-check-target-5277h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:07.232891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.232512 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs podName:46e97f49-b264-4a5e-8f66-9ae0ea812103 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:09.232500506 +0000 UTC m=+6.009132216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs") pod "network-metrics-daemon-jxjxg" (UID: "46e97f49-b264-4a5e-8f66-9ae0ea812103") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:07.232891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.232538 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd podName:159b39a6-357d-4362-800e-2bb1d2fca812 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:09.232522036 +0000 UTC m=+6.009153757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45rd" (UniqueName: "kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd") pod "network-check-target-5277h" (UID: "159b39a6-357d-4362-800e-2bb1d2fca812") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:07.659779 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.659736 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 14:54:05 +0000 UTC" deadline="2027-12-15 02:58:54.31651071 +0000 UTC" Jun 11 14:59:07.659779 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.659777 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13235h59m46.656738184s" Jun 11 14:59:07.669936 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.669911 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:07.670084 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.670048 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:07.670498 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.670448 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:07.670582 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.670543 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:07.670647 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.670609 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:07.670706 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:07.670677 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:07.718746 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.716059 2562 generic.go:356] "Generic (PLEG): container finished" podID="5d21059ae7ce928c107437710156b4ba" containerID="7f1b5b38bcf267df5af61906bfcca3589619f4ed6c24a357d159219438195bdf" exitCode=0 Jun 11 14:59:07.718746 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.716156 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" event={"ID":"5d21059ae7ce928c107437710156b4ba","Type":"ContainerDied","Data":"7f1b5b38bcf267df5af61906bfcca3589619f4ed6c24a357d159219438195bdf"} Jun 11 14:59:07.723065 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.722596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" event={"ID":"d985ea8e747104fff16cbaa5995c0307","Type":"ContainerStarted","Data":"1e5dc370440e3c25f9de517969f52c7c39bd948ba2831a9b51c8b3b6f9f3c4ae"} Jun 11 14:59:07.746601 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:07.746429 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-77.ec2.internal" podStartSLOduration=3.746407177 podStartE2EDuration="3.746407177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:07.745537708 +0000 UTC m=+4.522169430" watchObservedRunningTime="2026-06-11 14:59:07.746407177 +0000 UTC m=+4.523038900" Jun 11 14:59:08.751204 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:08.751164 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" event={"ID":"5d21059ae7ce928c107437710156b4ba","Type":"ContainerStarted","Data":"73698b4fcc9db6bf44369536e555c3a580db30bad9519abb49f6bad0978d9ebf"} Jun 11 14:59:08.769265 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:08.769169 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-77.ec2.internal" podStartSLOduration=4.769149018 podStartE2EDuration="4.769149018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:08.766975273 +0000 UTC m=+5.543606994" watchObservedRunningTime="2026-06-11 14:59:08.769149018 +0000 UTC m=+5.545780741" Jun 11 14:59:09.248263 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:09.248220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:09.248453 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:09.248294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:09.248453 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:09.248325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:09.248453 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.248447 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:09.248613 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.248512 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs podName:46e97f49-b264-4a5e-8f66-9ae0ea812103 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:13.248493458 +0000 UTC m=+10.025125175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs") pod "network-metrics-daemon-jxjxg" (UID: "46e97f49-b264-4a5e-8f66-9ae0ea812103") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:09.248949 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.248928 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:09.249033 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.248991 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret podName:b6572eff-fb22-4baa-b130-995f9284a608 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:13.248974192 +0000 UTC m=+10.025605909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret") pod "global-pull-secret-syncer-2zl8k" (UID: "b6572eff-fb22-4baa-b130-995f9284a608") : object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:09.249098 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.249077 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 14:59:09.249098 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.249097 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 14:59:09.249200 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.249110 2562 projected.go:196] Error preparing data for projected volume kube-api-access-c45rd for pod openshift-network-diagnostics/network-check-target-5277h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:09.249200 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.249151 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd podName:159b39a6-357d-4362-800e-2bb1d2fca812 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:13.249137213 +0000 UTC m=+10.025768919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45rd" (UniqueName: "kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd") pod "network-check-target-5277h" (UID: "159b39a6-357d-4362-800e-2bb1d2fca812") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:09.669062 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:09.669030 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:09.669238 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.669169 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:09.670024 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:09.669767 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:09.670024 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.669860 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:09.670024 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:09.669888 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:09.670024 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:09.669952 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:11.669581 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:11.669124 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:11.669581 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:11.669124 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:11.669581 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:11.669247 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:11.669581 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:11.669311 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:11.669581 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:11.669131 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:11.669581 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:11.669403 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:13.284393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:13.284457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:13.284489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.284542 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.284607 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret podName:b6572eff-fb22-4baa-b130-995f9284a608 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:21.284589623 +0000 UTC m=+18.061221345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret") pod "global-pull-secret-syncer-2zl8k" (UID: "b6572eff-fb22-4baa-b130-995f9284a608") : object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.284607 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.284644 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs podName:46e97f49-b264-4a5e-8f66-9ae0ea812103 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:21.284633945 +0000 UTC m=+18.061265644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs") pod "network-metrics-daemon-jxjxg" (UID: "46e97f49-b264-4a5e-8f66-9ae0ea812103") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.284677 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.284692 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.284704 2562 projected.go:196] Error preparing data for projected volume kube-api-access-c45rd for pod openshift-network-diagnostics/network-check-target-5277h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:13.284787 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.284758 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd podName:159b39a6-357d-4362-800e-2bb1d2fca812 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:21.284744591 +0000 UTC m=+18.061376303 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45rd" (UniqueName: "kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd") pod "network-check-target-5277h" (UID: "159b39a6-357d-4362-800e-2bb1d2fca812") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:13.671752 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:13.670968 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:13.671752 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.671077 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:13.671752 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:13.671455 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:13.671752 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.671566 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:13.671752 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:13.671625 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:13.671752 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:13.671698 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:15.670001 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:15.669964 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:15.670538 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:15.670095 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:15.670538 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:15.670410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:15.670656 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:15.670529 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:15.670656 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:15.670598 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:15.670765 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:15.670737 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:17.669864 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:17.669664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:17.670314 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:17.669709 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:17.670314 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:17.669959 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:17.670314 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:17.669746 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:17.670314 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:17.670049 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:17.670314 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:17.670126 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:19.669610 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:19.669578 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:19.670105 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:19.669578 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:19.670105 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:19.669578 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:19.670105 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:19.669754 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:19.670105 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:19.669803 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:19.670105 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:19.669894 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:21.348411 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:21.348361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:21.348411 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:21.348414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:21.348465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.348553 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.348571 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.348582 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.348598 2562 projected.go:196] Error preparing data for projected volume kube-api-access-c45rd for pod openshift-network-diagnostics/network-check-target-5277h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.348572 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.348663 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret podName:b6572eff-fb22-4baa-b130-995f9284a608 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:37.348616131 +0000 UTC m=+34.125247830 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret") pod "global-pull-secret-syncer-2zl8k" (UID: "b6572eff-fb22-4baa-b130-995f9284a608") : object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.348682 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd podName:159b39a6-357d-4362-800e-2bb1d2fca812 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:37.348672025 +0000 UTC m=+34.125303730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45rd" (UniqueName: "kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd") pod "network-check-target-5277h" (UID: "159b39a6-357d-4362-800e-2bb1d2fca812") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:21.348988 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.348699 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs podName:46e97f49-b264-4a5e-8f66-9ae0ea812103 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:37.348689388 +0000 UTC m=+34.125321091 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs") pod "network-metrics-daemon-jxjxg" (UID: "46e97f49-b264-4a5e-8f66-9ae0ea812103") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:21.670083 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:21.669996 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:21.670248 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:21.669994 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:21.670248 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.670135 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:21.670248 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:21.670173 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:21.670412 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.670254 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:21.670412 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:21.670315 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:23.670033 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:23.669999 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:23.670522 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:23.670107 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:23.670522 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:23.670123 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:23.670522 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:23.670216 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:23.670522 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:23.670260 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:23.670522 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:23.670325 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:25.669632 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.669467 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:25.670177 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.669484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:25.670177 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:25.669741 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:25.670177 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:25.669801 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:25.670177 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.669512 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:25.670177 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:25.669879 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:25.785515 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.784821 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8758w" event={"ID":"151bfb0e-d6da-4843-8f17-b1a14999fd3c","Type":"ContainerStarted","Data":"6f50f3674c98e3e4a0df80331d2dc5308cdcdebc483b92557cf371303ed4db2d"} Jun 11 14:59:25.786791 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.786764 2562 generic.go:356] "Generic (PLEG): container finished" podID="b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7" containerID="3eeb24855fb4a2f4ced2f156ede010b0f021a286e44cb890437011d459c941d5" exitCode=0 Jun 11 14:59:25.786891 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.786849 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-529bv" event={"ID":"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7","Type":"ContainerDied","Data":"3eeb24855fb4a2f4ced2f156ede010b0f021a286e44cb890437011d459c941d5"} Jun 11 14:59:25.788424 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.788398 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" event={"ID":"f798fa81-f718-4b9c-84e8-568b640625de","Type":"ContainerStarted","Data":"2dc64e6b306cdb56dacecb99528dcddc2179e84c6cbff66c45ec8142f0b386fa"} Jun 11 14:59:25.789893 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.789868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fhks6" event={"ID":"96157a6e-8a7c-444c-807e-8967dc056298","Type":"ContainerStarted","Data":"368f03daffec56c8a8ff00fa30fc555b5077b286226a6bcc6548e8f5f4589b56"} Jun 11 14:59:25.791288 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.791263 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bjhzp" event={"ID":"d6f5d097-246f-4602-a126-182e5018d288","Type":"ContainerStarted","Data":"5c88070aed86c4775a8f2bb8be92a0b2f0ff567037638359b4d8290d742f5253"} Jun 11 14:59:25.792620 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.792602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-99vwj" event={"ID":"8c998f7d-b189-4843-a760-069af193cde6","Type":"ContainerStarted","Data":"dbf6fe6451cb44fbaeea40a2b8062fc3aba57b0d76619cc8bb4af38042325d2b"} Jun 11 14:59:25.794396 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.794378 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"3fbe07dd4a80e7b22e58e0b8ebbcf02fab9e9bb46df41baefa8901b82de126b4"} Jun 11 14:59:25.794481 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.794449 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"c00708a95f1ce5734862d5eb6e964a11496b2156fad9521466f1665655c69fbc"} Jun 11 14:59:25.795548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.795526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-thlvs" event={"ID":"b06bf046-e052-4cfc-902b-05f689ad740d","Type":"ContainerStarted","Data":"f0d893cd584f7b7c59e450332b8b22744e36430ab46b93adb7372c4c91c06af3"} Jun 11 14:59:25.821547 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.821505 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8758w" podStartSLOduration=4.259389204 podStartE2EDuration="22.821490617s" totalImagesPullingTime="18.562101413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:25.801674198 +0000 UTC m=+22.578305916" watchObservedRunningTime="2026-06-11 14:59:25.821490617 +0000 UTC m=+22.598122338" Jun 11 14:59:25.837545 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.837499 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bjhzp" podStartSLOduration=4.142951825 podStartE2EDuration="22.837483562s" totalImagesPullingTime="18.694531737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:25.837371324 +0000 UTC m=+22.614003089" watchObservedRunningTime="2026-06-11 14:59:25.837483562 +0000 UTC m=+22.614115282" Jun 11 14:59:25.850270 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.850220 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-99vwj" podStartSLOduration=4.153919764 podStartE2EDuration="22.8502001s" totalImagesPullingTime="18.696280336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:25.850008205 +0000 UTC m=+22.626639941" watchObservedRunningTime="2026-06-11 14:59:25.8502001 +0000 UTC m=+22.626831826" Jun 11 14:59:25.866191 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.866168 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:25.866853 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.866812 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-thlvs" podStartSLOduration=4.209637009 podStartE2EDuration="22.866798091s" totalImagesPullingTime="18.657161082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:25.866433786 +0000 UTC m=+22.643065509" watchObservedRunningTime="2026-06-11 14:59:25.866798091 +0000 UTC m=+22.643429811" Jun 11 14:59:25.883245 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:25.883204 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fhks6" podStartSLOduration=4.177720809 podStartE2EDuration="22.883188248s" totalImagesPullingTime="18.705467439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:25.882561271 +0000 UTC m=+22.659192993" watchObservedRunningTime="2026-06-11 14:59:25.883188248 +0000 UTC m=+22.659819968" Jun 11 14:59:26.722116 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.721954 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 14:59:26.799526 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.799491 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bkpmb" event={"ID":"095e3504-ec5b-48dd-9683-9d235ff3bbcc","Type":"ContainerStarted","Data":"dcab83f5f37a431785bb971da3fa9be8284d4458c44788a586c375add3f3f885"} Jun 11 14:59:26.802542 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.802523 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 14:59:26.802907 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.802883 2562 generic.go:356] "Generic (PLEG): container finished" podID="945a32d3-e116-4317-b5c7-23856c9645fb" containerID="3fbe07dd4a80e7b22e58e0b8ebbcf02fab9e9bb46df41baefa8901b82de126b4" exitCode=1 Jun 11 14:59:26.803042 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.802922 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerDied","Data":"3fbe07dd4a80e7b22e58e0b8ebbcf02fab9e9bb46df41baefa8901b82de126b4"} Jun 11 14:59:26.803042 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.803006 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"d8ef119dd9560acc6a2e9f99e81e8c8faf3422a09673ab2d1bed9c1701119a06"} Jun 11 14:59:26.803150 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.803061 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"97444efa4090e7f66e2c8d3b9ad88dd78104d15c0e94c3e25f8b0218f6dc5be7"} Jun 11 14:59:26.803150 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.803109 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"eb90325d270739b552923187b5a95cc141cc38f4dd9c8db10398dbc36bd2c68e"} Jun 11 14:59:26.803243 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.803158 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"baa4e7b78a702bd6ef64ffdaba043fc0645c4491a842e04af2f9cd94fa1ef9c4"} Jun 11 14:59:26.804924 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.804854 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" event={"ID":"f798fa81-f718-4b9c-84e8-568b640625de","Type":"ContainerStarted","Data":"a00f7253bd0d7dac70dc46d7c4648a6958bc49e95c6aa2e1bb2794289c7fa43b"} Jun 11 14:59:26.813934 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.813862 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bkpmb" podStartSLOduration=5.2537295 podStartE2EDuration="23.813847463s" totalImagesPullingTime="18.560117963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:26.81341151 +0000 UTC m=+23.590043230" watchObservedRunningTime="2026-06-11 14:59:26.813847463 +0000 UTC m=+23.590479186" Jun 11 14:59:26.912115 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.912080 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:26.912607 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:26.912585 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:27.661979 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:27.661867 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T14:59:26.722112594Z","UUID":"d22daece-0cee-4ce0-a772-52bf40c54ee0","Handler":null,"Name":"","Endpoint":""} Jun 11 14:59:27.663433 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:27.663410 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 14:59:27.663567 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:27.663455 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 14:59:27.669356 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:27.669218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:27.669356 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:27.669252 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:27.669356 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:27.669259 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:27.669356 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:27.669339 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:27.669618 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:27.669464 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:27.669618 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:27.669550 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:27.808000 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:27.807976 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8758w" Jun 11 14:59:28.812569 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:28.812443 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" event={"ID":"f798fa81-f718-4b9c-84e8-568b640625de","Type":"ContainerStarted","Data":"dbfbcebe2d3b5c09da32e771d7bbe6823c99b5ed37b67f8806f3503433679da5"} Jun 11 14:59:28.815991 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:28.815969 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 14:59:28.816413 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:28.816392 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"56c663137ea5e366c727822fbe69ffcdee287bc19ee6662d94b69386e06981bc"} Jun 11 14:59:28.829522 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:28.829474 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z6d6q" podStartSLOduration=5.164504301 podStartE2EDuration="25.829453875s" totalImagesPullingTime="20.664949574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:28.82917264 +0000 UTC m=+25.605804361" watchObservedRunningTime="2026-06-11 14:59:28.829453875 +0000 UTC m=+25.606085596" Jun 11 14:59:29.669411 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:29.669383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:29.669411 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:29.669411 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:29.669626 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:29.669383 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:29.669626 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:29.669518 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:29.669626 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:29.669595 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:29.669742 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:29.669683 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:30.822941 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:30.822650 2562 generic.go:356] "Generic (PLEG): container finished" podID="b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7" containerID="ea7f0b4af643dd9639d2e8493c99da381dde09481fd577fbcded26763a18c6bd" exitCode=0 Jun 11 14:59:30.822941 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:30.822731 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-529bv" event={"ID":"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7","Type":"ContainerDied","Data":"ea7f0b4af643dd9639d2e8493c99da381dde09481fd577fbcded26763a18c6bd"} Jun 11 14:59:30.826435 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:30.826415 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 14:59:30.826792 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:30.826770 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"ef4958560b32150e9d6ba7da7f80963c388beebbfb74bb9aafc4e737a9eab1f8"} Jun 11 14:59:30.827054 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:30.827031 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:30.827149 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:30.827091 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:30.827189 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:30.827159 2562 scope.go:117] "RemoveContainer" containerID="3fbe07dd4a80e7b22e58e0b8ebbcf02fab9e9bb46df41baefa8901b82de126b4" Jun 11 14:59:30.843682 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:30.843665 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:31.669247 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:31.669217 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:31.669407 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:31.669217 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:31.669407 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:31.669327 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:31.669476 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:31.669406 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:31.669476 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:31.669225 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:31.669541 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:31.669475 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:31.832365 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:31.832342 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 14:59:31.832801 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:31.832781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" event={"ID":"945a32d3-e116-4317-b5c7-23856c9645fb","Type":"ContainerStarted","Data":"350917bb5feaf2058b3d3621855fe88d32eba548081151c34ff00c6453d7c0b4"} Jun 11 14:59:31.833012 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:31.832994 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:31.847989 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:31.847969 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 14:59:31.863442 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:31.863401 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" podStartSLOduration=10.124900814 podStartE2EDuration="28.86338674s" totalImagesPullingTime="18.738485926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:31.86120077 +0000 UTC m=+28.637832492" watchObservedRunningTime="2026-06-11 14:59:31.86338674 +0000 UTC m=+28.640018463" Jun 11 14:59:32.469933 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:32.469757 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2zl8k"] Jun 11 14:59:32.470140 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:32.470049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:32.470206 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:32.470159 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:32.472784 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:32.472759 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jxjxg"] Jun 11 14:59:32.472929 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:32.472900 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:32.473001 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:32.472987 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:32.473329 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:32.473307 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5277h"] Jun 11 14:59:32.473422 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:32.473391 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:32.473484 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:32.473465 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:32.836856 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:32.836820 2562 generic.go:356] "Generic (PLEG): container finished" podID="b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7" containerID="07bfaa6addd97aa093ec8709e0ab5539f69b30afa1b29adf7b9d176f47e9d87f" exitCode=0 Jun 11 14:59:32.837430 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:32.836907 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-529bv" event={"ID":"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7","Type":"ContainerDied","Data":"07bfaa6addd97aa093ec8709e0ab5539f69b30afa1b29adf7b9d176f47e9d87f"} Jun 11 14:59:34.669749 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:34.669701 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:34.670252 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:34.669810 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:34.670252 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:34.669818 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:34.670252 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:34.669840 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:34.670252 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:34.669907 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:34.670252 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:34.669975 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:34.845875 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:34.845846 2562 generic.go:356] "Generic (PLEG): container finished" podID="b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7" containerID="1e3a0cc66382b26dc49ded0e6d47b379e9f3e0f1e9771791a48a31e9a220b38e" exitCode=0 Jun 11 14:59:34.846019 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:34.845903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-529bv" event={"ID":"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7","Type":"ContainerDied","Data":"1e3a0cc66382b26dc49ded0e6d47b379e9f3e0f1e9771791a48a31e9a220b38e"} Jun 11 14:59:36.669748 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:36.669490 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:36.670246 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:36.669531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:36.670246 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:36.669859 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2zl8k" podUID="b6572eff-fb22-4baa-b130-995f9284a608" Jun 11 14:59:36.670246 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:36.669538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:36.670246 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:36.669931 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jxjxg" podUID="46e97f49-b264-4a5e-8f66-9ae0ea812103" Jun 11 14:59:36.670246 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:36.669989 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5277h" podUID="159b39a6-357d-4362-800e-2bb1d2fca812" Jun 11 14:59:37.374726 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:37.374687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:37.374913 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:37.374765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:37.374913 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:37.374791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:37.374913 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:37.374843 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:37.374913 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:37.374901 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:37.374913 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:37.374913 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret podName:b6572eff-fb22-4baa-b130-995f9284a608 nodeName:}" failed. No retries permitted until 2026-06-11 15:00:09.374896232 +0000 UTC m=+66.151527930 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret") pod "global-pull-secret-syncer-2zl8k" (UID: "b6572eff-fb22-4baa-b130-995f9284a608") : object "kube-system"/"original-pull-secret" not registered Jun 11 14:59:37.375146 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:37.374919 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 14:59:37.375146 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:37.374941 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 14:59:37.375146 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:37.374954 2562 projected.go:196] Error preparing data for projected volume kube-api-access-c45rd for pod openshift-network-diagnostics/network-check-target-5277h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:37.375146 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:37.374960 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs podName:46e97f49-b264-4a5e-8f66-9ae0ea812103 nodeName:}" failed. No retries permitted until 2026-06-11 15:00:09.374941778 +0000 UTC m=+66.151573497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs") pod "network-metrics-daemon-jxjxg" (UID: "46e97f49-b264-4a5e-8f66-9ae0ea812103") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 14:59:37.375146 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:37.375002 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd podName:159b39a6-357d-4362-800e-2bb1d2fca812 nodeName:}" failed. No retries permitted until 2026-06-11 15:00:09.374988798 +0000 UTC m=+66.151620499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45rd" (UniqueName: "kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd") pod "network-check-target-5277h" (UID: "159b39a6-357d-4362-800e-2bb1d2fca812") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 14:59:38.552534 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.552503 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-77.ec2.internal" event="NodeReady" Jun 11 14:59:38.552995 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.552671 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 14:59:38.598601 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.598568 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zvgk5"] Jun 11 14:59:38.637963 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.637931 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v95cn"] Jun 11 14:59:38.638105 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.638002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.640929 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.640909 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 14:59:38.640929 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.640924 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 14:59:38.641106 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.640960 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmhvb\"" Jun 11 14:59:38.656061 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.656042 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvgk5"] Jun 11 14:59:38.656152 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.656074 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v95cn"] Jun 11 14:59:38.656152 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.656139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v95cn" Jun 11 14:59:38.658809 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.658791 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 14:59:38.659084 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.659066 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 14:59:38.659084 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.659085 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 14:59:38.659291 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.659097 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hmb6r\"" Jun 11 14:59:38.669244 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.669225 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 14:59:38.669372 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.669315 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 14:59:38.669462 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.669445 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 14:59:38.671894 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.671874 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 14:59:38.671894 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.671879 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 14:59:38.672035 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.671900 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 14:59:38.672035 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.671910 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p4p2b\"" Jun 11 14:59:38.672035 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.671884 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 14:59:38.672330 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.672313 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdl6h\"" Jun 11 14:59:38.784456 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.784422 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94eacb66-220d-4890-90a3-5840d70579ae-config-volume\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.784648 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.784465 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2-cert\") pod \"ingress-canary-v95cn\" (UID: \"c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2\") " pod="openshift-ingress-canary/ingress-canary-v95cn" Jun 11 14:59:38.784648 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.784556 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94eacb66-220d-4890-90a3-5840d70579ae-tmp-dir\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.784648 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.784582 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.784648 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.784603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzxsk\" (UniqueName: \"kubernetes.io/projected/94eacb66-220d-4890-90a3-5840d70579ae-kube-api-access-xzxsk\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.784859 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.784651 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rnnc\" (UniqueName: \"kubernetes.io/projected/c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2-kube-api-access-6rnnc\") pod \"ingress-canary-v95cn\" (UID: \"c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2\") " pod="openshift-ingress-canary/ingress-canary-v95cn" Jun 11 14:59:38.885064 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.885044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94eacb66-220d-4890-90a3-5840d70579ae-tmp-dir\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.885163 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.885073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.885163 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.885092 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzxsk\" (UniqueName: \"kubernetes.io/projected/94eacb66-220d-4890-90a3-5840d70579ae-kube-api-access-xzxsk\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.885163 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.885145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rnnc\" (UniqueName: \"kubernetes.io/projected/c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2-kube-api-access-6rnnc\") pod \"ingress-canary-v95cn\" (UID: \"c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2\") " pod="openshift-ingress-canary/ingress-canary-v95cn" Jun 11 14:59:38.885303 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.885184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94eacb66-220d-4890-90a3-5840d70579ae-config-volume\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.885303 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:38.885205 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 14:59:38.885303 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.885219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2-cert\") pod \"ingress-canary-v95cn\" (UID: \"c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2\") " pod="openshift-ingress-canary/ingress-canary-v95cn" Jun 11 14:59:38.885303 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:38.885265 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls podName:94eacb66-220d-4890-90a3-5840d70579ae nodeName:}" failed. No retries permitted until 2026-06-11 14:59:39.385245682 +0000 UTC m=+36.161877405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls") pod "dns-default-zvgk5" (UID: "94eacb66-220d-4890-90a3-5840d70579ae") : secret "dns-default-metrics-tls" not found Jun 11 14:59:38.885457 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.885432 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94eacb66-220d-4890-90a3-5840d70579ae-tmp-dir\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.885890 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.885866 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/94eacb66-220d-4890-90a3-5840d70579ae-config-volume\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.889150 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.889128 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2-cert\") pod \"ingress-canary-v95cn\" (UID: \"c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2\") " pod="openshift-ingress-canary/ingress-canary-v95cn" Jun 11 14:59:38.894121 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.894100 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzxsk\" (UniqueName: \"kubernetes.io/projected/94eacb66-220d-4890-90a3-5840d70579ae-kube-api-access-xzxsk\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:38.894412 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.894380 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rnnc\" (UniqueName: \"kubernetes.io/projected/c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2-kube-api-access-6rnnc\") pod \"ingress-canary-v95cn\" (UID: \"c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2\") " pod="openshift-ingress-canary/ingress-canary-v95cn" Jun 11 14:59:38.963676 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:38.963649 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v95cn" Jun 11 14:59:39.135815 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:39.135787 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v95cn"] Jun 11 14:59:39.138928 ip-10-0-140-77 kubenswrapper[2562]: W0611 14:59:39.138906 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc828ebd1_aa90_4f02_b6c3_c3ecdd97a4b2.slice/crio-3819ffb51e541d33a7e41ec639f68236f63e33fb6d8a963aca2df76940130548 WatchSource:0}: Error finding container 3819ffb51e541d33a7e41ec639f68236f63e33fb6d8a963aca2df76940130548: Status 404 returned error can't find the container with id 3819ffb51e541d33a7e41ec639f68236f63e33fb6d8a963aca2df76940130548 Jun 11 14:59:39.388472 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:39.388441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:39.388646 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:39.388598 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 14:59:39.388697 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:39.388663 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls podName:94eacb66-220d-4890-90a3-5840d70579ae nodeName:}" failed. No retries permitted until 2026-06-11 14:59:40.38864795 +0000 UTC m=+37.165279648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls") pod "dns-default-zvgk5" (UID: "94eacb66-220d-4890-90a3-5840d70579ae") : secret "dns-default-metrics-tls" not found Jun 11 14:59:39.860241 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:39.860202 2562 generic.go:356] "Generic (PLEG): container finished" podID="b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7" containerID="ed702e65fa62511dc16151db70a1653e7d5146af48666252d8a0a6f2745a0a68" exitCode=0 Jun 11 14:59:39.860688 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:39.860290 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-529bv" event={"ID":"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7","Type":"ContainerDied","Data":"ed702e65fa62511dc16151db70a1653e7d5146af48666252d8a0a6f2745a0a68"} Jun 11 14:59:39.862302 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:39.862275 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v95cn" event={"ID":"c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2","Type":"ContainerStarted","Data":"3819ffb51e541d33a7e41ec639f68236f63e33fb6d8a963aca2df76940130548"} Jun 11 14:59:40.397497 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.397259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:40.397757 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:40.397422 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 14:59:40.397757 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:40.397655 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls podName:94eacb66-220d-4890-90a3-5840d70579ae nodeName:}" failed. No retries permitted until 2026-06-11 14:59:42.397630067 +0000 UTC m=+39.174261769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls") pod "dns-default-zvgk5" (UID: "94eacb66-220d-4890-90a3-5840d70579ae") : secret "dns-default-metrics-tls" not found Jun 11 14:59:40.867859 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.867822 2562 generic.go:356] "Generic (PLEG): container finished" podID="b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7" containerID="54de88fec240a4328c213d641d566724ca5298c98d2441dd42513d4962ea1444" exitCode=0 Jun 11 14:59:40.868130 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.867871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-529bv" event={"ID":"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7","Type":"ContainerDied","Data":"54de88fec240a4328c213d641d566724ca5298c98d2441dd42513d4962ea1444"} Jun 11 14:59:40.967051 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.967019 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5"] Jun 11 14:59:40.979813 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.979779 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5"] Jun 11 14:59:40.979960 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.979902 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:40.982359 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.982339 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 11 14:59:40.982797 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.982782 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 14:59:40.984367 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.984246 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 14:59:40.984367 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.984262 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 11 14:59:40.984367 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:40.984321 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7rxgc\"" Jun 11 14:59:41.071176 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.071143 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58696fc5c4-rjdjt"] Jun 11 14:59:41.093483 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.093459 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-868f598b87-7svzh"] Jun 11 14:59:41.093629 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.093582 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.099832 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.096763 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 11 14:59:41.099832 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.096792 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8h69h\"" Jun 11 14:59:41.099832 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.097337 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 11 14:59:41.099832 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.097356 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 11 14:59:41.102348 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.102315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/848c9644-faa6-430e-8cc1-b8b3f451f7bd-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.102468 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.102431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.102646 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.102512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq5g5\" (UniqueName: \"kubernetes.io/projected/848c9644-faa6-430e-8cc1-b8b3f451f7bd-kube-api-access-vq5g5\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.103048 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.102885 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 11 14:59:41.111412 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.111391 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-868f598b87-7svzh"] Jun 11 14:59:41.111508 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.111427 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58696fc5c4-rjdjt"] Jun 11 14:59:41.111546 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.111513 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.114476 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.114426 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 14:59:41.114476 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.114462 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rnwdz\"" Jun 11 14:59:41.114650 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.114542 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 11 14:59:41.114650 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.114557 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 11 14:59:41.114779 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.114693 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 11 14:59:41.114836 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.114818 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 11 14:59:41.114936 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.114919 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 14:59:41.203364 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.203532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203373 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-default-certificate\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.203532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203413 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8f84c27-943e-4dd0-83e0-4f24a95114bf-ca-trust-extracted\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.203532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203429 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-certificates\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.203532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203445 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.203532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203463 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-image-registry-private-configuration\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.203532 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203490 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.203532 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.203525 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:41.203798 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.203606 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls podName:848c9644-faa6-430e-8cc1-b8b3f451f7bd nodeName:}" failed. No retries permitted until 2026-06-11 14:59:41.703583766 +0000 UTC m=+38.480215483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-wq5z5" (UID: "848c9644-faa6-430e-8cc1-b8b3f451f7bd") : secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:41.203798 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.203798 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq5g5\" (UniqueName: \"kubernetes.io/projected/848c9644-faa6-430e-8cc1-b8b3f451f7bd-kube-api-access-vq5g5\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.203798 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m77xl\" (UniqueName: \"kubernetes.io/projected/5c39500a-f1fc-4a5f-b922-1ff7217234b0-kube-api-access-m77xl\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.203798 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203743 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-installation-pull-secrets\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.203798 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/848c9644-faa6-430e-8cc1-b8b3f451f7bd-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.204029 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203848 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-bound-sa-token\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.204029 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203900 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9rb2\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-kube-api-access-z9rb2\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.204029 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-stats-auth\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.204029 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.203972 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-trusted-ca\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.204566 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.204540 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/848c9644-faa6-430e-8cc1-b8b3f451f7bd-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.212447 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.212421 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq5g5\" (UniqueName: \"kubernetes.io/projected/848c9644-faa6-430e-8cc1-b8b3f451f7bd-kube-api-access-vq5g5\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.304703 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-default-certificate\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.304904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8f84c27-943e-4dd0-83e0-4f24a95114bf-ca-trust-extracted\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.304904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-certificates\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.304904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.304904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304850 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-image-registry-private-configuration\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.304904 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304888 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304910 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m77xl\" (UniqueName: \"kubernetes.io/projected/5c39500a-f1fc-4a5f-b922-1ff7217234b0-kube-api-access-m77xl\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.304940 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.304966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-installation-pull-secrets\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.305024 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:41.805003618 +0000 UTC m=+38.581635338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : secret "router-metrics-certs-default" not found Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.305065 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:41.805045236 +0000 UTC m=+38.581676958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : configmap references non-existent config key: service-ca.crt Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.305096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-bound-sa-token\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.305129 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9rb2\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-kube-api-access-z9rb2\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.305172 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.305158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-stats-auth\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.305548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.305184 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8f84c27-943e-4dd0-83e0-4f24a95114bf-ca-trust-extracted\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.305548 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.305189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-trusted-ca\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.305659 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.305542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-certificates\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.305659 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.305613 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 14:59:41.305659 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.305629 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58696fc5c4-rjdjt: secret "image-registry-tls" not found Jun 11 14:59:41.305814 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.305685 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls podName:b8f84c27-943e-4dd0-83e0-4f24a95114bf nodeName:}" failed. No retries permitted until 2026-06-11 14:59:41.805666332 +0000 UTC m=+38.582298030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls") pod "image-registry-58696fc5c4-rjdjt" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf") : secret "image-registry-tls" not found Jun 11 14:59:41.306199 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.306173 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-trusted-ca\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.307774 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.307744 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-installation-pull-secrets\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.307899 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.307818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-default-certificate\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.308099 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.308080 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-stats-auth\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.308171 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.308129 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-image-registry-private-configuration\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.313570 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.313542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9rb2\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-kube-api-access-z9rb2\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.313659 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.313642 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-bound-sa-token\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.314036 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.314018 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m77xl\" (UniqueName: \"kubernetes.io/projected/5c39500a-f1fc-4a5f-b922-1ff7217234b0-kube-api-access-m77xl\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.709094 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.709061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:41.709242 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.709204 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:41.709312 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.709302 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls podName:848c9644-faa6-430e-8cc1-b8b3f451f7bd nodeName:}" failed. No retries permitted until 2026-06-11 14:59:42.709287419 +0000 UTC m=+39.485919120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-wq5z5" (UID: "848c9644-faa6-430e-8cc1-b8b3f451f7bd") : secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:41.810383 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.810343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.810383 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.810382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:41.810569 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.810398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:41.810569 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.810484 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 14:59:41.810569 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.810488 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 14:59:41.810569 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.810563 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:42.810545375 +0000 UTC m=+39.587177091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : secret "router-metrics-certs-default" not found Jun 11 14:59:41.810569 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.810497 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58696fc5c4-rjdjt: secret "image-registry-tls" not found Jun 11 14:59:41.810762 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.810611 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:42.810600319 +0000 UTC m=+39.587232034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : configmap references non-existent config key: service-ca.crt Jun 11 14:59:41.810762 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:41.810629 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls podName:b8f84c27-943e-4dd0-83e0-4f24a95114bf nodeName:}" failed. No retries permitted until 2026-06-11 14:59:42.810621421 +0000 UTC m=+39.587253137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls") pod "image-registry-58696fc5c4-rjdjt" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf") : secret "image-registry-tls" not found Jun 11 14:59:41.873780 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.873745 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-529bv" event={"ID":"b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7","Type":"ContainerStarted","Data":"65b3b0a358b12809e777c8272fdba1d70342232f4cc8d26f13d99a451ae49602"} Jun 11 14:59:41.875223 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.875199 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v95cn" event={"ID":"c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2","Type":"ContainerStarted","Data":"a711188ff003251c039ad716d80c6efb92efe9550129a65376add3784dc50486"} Jun 11 14:59:41.895191 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.895152 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-529bv" podStartSLOduration=9.795871626 podStartE2EDuration="38.895138256s" totalImagesPullingTime="29.09926663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:41.893193153 +0000 UTC m=+38.669824873" watchObservedRunningTime="2026-06-11 14:59:41.895138256 +0000 UTC m=+38.671770025" Jun 11 14:59:41.906689 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:41.906645 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v95cn" podStartSLOduration=2.22379056 podStartE2EDuration="3.906632123s" totalImagesPullingTime="1.682841563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 14:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 14:59:41.906416312 +0000 UTC m=+38.683048033" watchObservedRunningTime="2026-06-11 14:59:41.906632123 +0000 UTC m=+38.683263835" Jun 11 14:59:42.415771 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:42.415734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:42.415930 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.415884 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 14:59:42.415974 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.415951 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls podName:94eacb66-220d-4890-90a3-5840d70579ae nodeName:}" failed. No retries permitted until 2026-06-11 14:59:46.415934403 +0000 UTC m=+43.192566105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls") pod "dns-default-zvgk5" (UID: "94eacb66-220d-4890-90a3-5840d70579ae") : secret "dns-default-metrics-tls" not found Jun 11 14:59:42.719097 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:42.719003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:42.719239 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.719144 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:42.719239 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.719203 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls podName:848c9644-faa6-430e-8cc1-b8b3f451f7bd nodeName:}" failed. No retries permitted until 2026-06-11 14:59:44.719188883 +0000 UTC m=+41.495820587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-wq5z5" (UID: "848c9644-faa6-430e-8cc1-b8b3f451f7bd") : secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:42.820252 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:42.820214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:42.820252 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:42.820252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:42.820441 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:42.820285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:42.820441 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.820370 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 14:59:42.820441 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.820374 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 14:59:42.820441 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.820419 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58696fc5c4-rjdjt: secret "image-registry-tls" not found Jun 11 14:59:42.820441 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.820437 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:44.820424695 +0000 UTC m=+41.597056407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : configmap references non-existent config key: service-ca.crt Jun 11 14:59:42.820599 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.820452 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls podName:b8f84c27-943e-4dd0-83e0-4f24a95114bf nodeName:}" failed. No retries permitted until 2026-06-11 14:59:44.820445832 +0000 UTC m=+41.597077531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls") pod "image-registry-58696fc5c4-rjdjt" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf") : secret "image-registry-tls" not found Jun 11 14:59:42.820599 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:42.820464 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:44.820458974 +0000 UTC m=+41.597090685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : secret "router-metrics-certs-default" not found Jun 11 14:59:44.737861 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:44.737820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:44.738282 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:44.737988 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:44.738282 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:44.738057 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls podName:848c9644-faa6-430e-8cc1-b8b3f451f7bd nodeName:}" failed. No retries permitted until 2026-06-11 14:59:48.738041064 +0000 UTC m=+45.514672762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-wq5z5" (UID: "848c9644-faa6-430e-8cc1-b8b3f451f7bd") : secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:44.838654 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:44.838621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:44.838791 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:44.838662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:44.838791 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:44.838687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:44.838891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:44.838794 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 14:59:44.838891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:44.838824 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58696fc5c4-rjdjt: secret "image-registry-tls" not found Jun 11 14:59:44.838891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:44.838791 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 14:59:44.838891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:44.838869 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:48.838852684 +0000 UTC m=+45.615484383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : configmap references non-existent config key: service-ca.crt Jun 11 14:59:44.839055 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:44.838892 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls podName:b8f84c27-943e-4dd0-83e0-4f24a95114bf nodeName:}" failed. No retries permitted until 2026-06-11 14:59:48.838885239 +0000 UTC m=+45.615516938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls") pod "image-registry-58696fc5c4-rjdjt" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf") : secret "image-registry-tls" not found Jun 11 14:59:44.839055 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:44.838904 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:48.838899223 +0000 UTC m=+45.615530922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : secret "router-metrics-certs-default" not found Jun 11 14:59:46.450167 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:46.450124 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:46.450597 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:46.450284 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 14:59:46.450597 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:46.450349 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls podName:94eacb66-220d-4890-90a3-5840d70579ae nodeName:}" failed. No retries permitted until 2026-06-11 14:59:54.450333279 +0000 UTC m=+51.226964978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls") pod "dns-default-zvgk5" (UID: "94eacb66-220d-4890-90a3-5840d70579ae") : secret "dns-default-metrics-tls" not found Jun 11 14:59:47.969284 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:47.969256 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bjhzp_d6f5d097-246f-4602-a126-182e5018d288/dns-node-resolver/0.log" Jun 11 14:59:48.765016 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.764985 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-99vwj_8c998f7d-b189-4843-a760-069af193cde6/node-ca/0.log" Jun 11 14:59:48.767300 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.767279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:48.767417 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.767404 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:48.767470 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.767460 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls podName:848c9644-faa6-430e-8cc1-b8b3f451f7bd nodeName:}" failed. No retries permitted until 2026-06-11 14:59:56.767446879 +0000 UTC m=+53.544078582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-wq5z5" (UID: "848c9644-faa6-430e-8cc1-b8b3f451f7bd") : secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:48.831621 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.831594 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6nv4q"] Jun 11 14:59:48.860118 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.860091 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6nv4q"] Jun 11 14:59:48.860236 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.860215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.864071 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.864045 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 14:59:48.864183 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.864082 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p65kp\"" Jun 11 14:59:48.864656 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.864638 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 14:59:48.864708 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.864648 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 14:59:48.864783 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.864760 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 14:59:48.868586 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.868567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/595146ac-1555-4472-8500-1f4b84565a05-crio-socket\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.868679 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.868604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:48.868679 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.868666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:48.868824 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.868698 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 14:59:48.868824 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.868764 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:56.868749552 +0000 UTC m=+53.645381251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : secret "router-metrics-certs-default" not found Jun 11 14:59:48.868824 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.868695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/595146ac-1555-4472-8500-1f4b84565a05-data-volume\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.868824 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.868797 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 14:59:48.868824 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.868811 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58696fc5c4-rjdjt: secret "image-registry-tls" not found Jun 11 14:59:48.869073 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.868825 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.869073 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.868867 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls podName:b8f84c27-943e-4dd0-83e0-4f24a95114bf nodeName:}" failed. No retries permitted until 2026-06-11 14:59:56.868851023 +0000 UTC m=+53.645482742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls") pod "image-registry-58696fc5c4-rjdjt" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf") : secret "image-registry-tls" not found Jun 11 14:59:48.869073 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.868905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:48.869073 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.868946 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxhf7\" (UniqueName: \"kubernetes.io/projected/595146ac-1555-4472-8500-1f4b84565a05-kube-api-access-qxhf7\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.869073 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.869004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/595146ac-1555-4472-8500-1f4b84565a05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.869073 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.869032 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:56.869017671 +0000 UTC m=+53.645649374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : configmap references non-existent config key: service-ca.crt Jun 11 14:59:48.969526 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.969493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/595146ac-1555-4472-8500-1f4b84565a05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.969983 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.969540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/595146ac-1555-4472-8500-1f4b84565a05-crio-socket\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.969983 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.969607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/595146ac-1555-4472-8500-1f4b84565a05-data-volume\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.969983 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.969650 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.969983 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.969741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxhf7\" (UniqueName: \"kubernetes.io/projected/595146ac-1555-4472-8500-1f4b84565a05-kube-api-access-qxhf7\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.969983 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.969811 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 14:59:48.969983 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:48.969873 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls podName:595146ac-1555-4472-8500-1f4b84565a05 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:49.469859204 +0000 UTC m=+46.246490903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6nv4q" (UID: "595146ac-1555-4472-8500-1f4b84565a05") : secret "insights-runtime-extractor-tls" not found Jun 11 14:59:48.969983 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.969814 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/595146ac-1555-4472-8500-1f4b84565a05-crio-socket\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.970214 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.969987 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/595146ac-1555-4472-8500-1f4b84565a05-data-volume\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.970214 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.970100 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/595146ac-1555-4472-8500-1f4b84565a05-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:48.986820 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:48.986796 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxhf7\" (UniqueName: \"kubernetes.io/projected/595146ac-1555-4472-8500-1f4b84565a05-kube-api-access-qxhf7\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:49.473773 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:49.473703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:49.473940 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:49.473858 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 14:59:49.473940 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:49.473931 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls podName:595146ac-1555-4472-8500-1f4b84565a05 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:50.473914191 +0000 UTC m=+47.250545893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6nv4q" (UID: "595146ac-1555-4472-8500-1f4b84565a05") : secret "insights-runtime-extractor-tls" not found Jun 11 14:59:49.764481 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:49.764408 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v95cn_c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2/serve-healthcheck-canary/0.log" Jun 11 14:59:50.482748 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:50.482691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:50.483126 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:50.482836 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 14:59:50.483126 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:50.482899 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls podName:595146ac-1555-4472-8500-1f4b84565a05 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:52.482883676 +0000 UTC m=+49.259515375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6nv4q" (UID: "595146ac-1555-4472-8500-1f4b84565a05") : secret "insights-runtime-extractor-tls" not found Jun 11 14:59:52.497376 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:52.497338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:52.497765 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:52.497508 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 14:59:52.497765 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:52.497584 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls podName:595146ac-1555-4472-8500-1f4b84565a05 nodeName:}" failed. No retries permitted until 2026-06-11 14:59:56.497565506 +0000 UTC m=+53.274197204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6nv4q" (UID: "595146ac-1555-4472-8500-1f4b84565a05") : secret "insights-runtime-extractor-tls" not found Jun 11 14:59:54.513625 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:54.513589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 14:59:54.514091 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:54.513747 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 14:59:54.514091 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:54.513799 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls podName:94eacb66-220d-4890-90a3-5840d70579ae nodeName:}" failed. No retries permitted until 2026-06-11 15:00:10.513785288 +0000 UTC m=+67.290416986 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls") pod "dns-default-zvgk5" (UID: "94eacb66-220d-4890-90a3-5840d70579ae") : secret "dns-default-metrics-tls" not found Jun 11 14:59:56.528199 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:56.528167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 14:59:56.528589 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.528296 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 14:59:56.528589 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.528375 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls podName:595146ac-1555-4472-8500-1f4b84565a05 nodeName:}" failed. No retries permitted until 2026-06-11 15:00:04.528360412 +0000 UTC m=+61.304992112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6nv4q" (UID: "595146ac-1555-4472-8500-1f4b84565a05") : secret "insights-runtime-extractor-tls" not found Jun 11 14:59:56.831341 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:56.831256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 14:59:56.831494 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.831411 2562 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:56.831538 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.831494 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls podName:848c9644-faa6-430e-8cc1-b8b3f451f7bd nodeName:}" failed. No retries permitted until 2026-06-11 15:00:12.831469153 +0000 UTC m=+69.608100869 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-wq5z5" (UID: "848c9644-faa6-430e-8cc1-b8b3f451f7bd") : secret "cluster-monitoring-operator-tls" not found Jun 11 14:59:56.932611 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:56.932581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls\") pod \"image-registry-58696fc5c4-rjdjt\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 14:59:56.932782 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:56.932658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 14:59:56.932782 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.932751 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 14:59:56.932782 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.932768 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58696fc5c4-rjdjt: secret "image-registry-tls" not found Jun 11 14:59:56.932891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.932811 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 14:59:56.932891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.932822 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls podName:b8f84c27-943e-4dd0-83e0-4f24a95114bf nodeName:}" failed. No retries permitted until 2026-06-11 15:00:12.93280328 +0000 UTC m=+69.709434994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls") pod "image-registry-58696fc5c4-rjdjt" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf") : secret "image-registry-tls" not found Jun 11 14:59:56.932891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.932839 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 15:00:12.93283003 +0000 UTC m=+69.709461731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : configmap references non-existent config key: service-ca.crt Jun 11 14:59:56.932891 ip-10-0-140-77 kubenswrapper[2562]: E0611 14:59:56.932853 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs podName:5c39500a-f1fc-4a5f-b922-1ff7217234b0 nodeName:}" failed. No retries permitted until 2026-06-11 15:00:12.932845177 +0000 UTC m=+69.709476877 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs") pod "router-default-868f598b87-7svzh" (UID: "5c39500a-f1fc-4a5f-b922-1ff7217234b0") : secret "router-metrics-certs-default" not found Jun 11 14:59:56.932891 ip-10-0-140-77 kubenswrapper[2562]: I0611 14:59:56.932753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:03.851662 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:03.851636 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h9k9f" Jun 11 15:00:04.601654 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:04.601615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 15:00:04.603892 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:04.603862 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/595146ac-1555-4472-8500-1f4b84565a05-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6nv4q\" (UID: \"595146ac-1555-4472-8500-1f4b84565a05\") " pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 15:00:04.771727 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:04.771687 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p65kp\"" Jun 11 15:00:04.779555 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:04.779531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6nv4q" Jun 11 15:00:04.904407 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:04.902126 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6nv4q"] Jun 11 15:00:04.907164 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:04.907140 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595146ac_1555_4472_8500_1f4b84565a05.slice/crio-f377dc8c3a3de3bc602471f006f6c0415c8853e55b56a2ed14019c5165cd4278 WatchSource:0}: Error finding container f377dc8c3a3de3bc602471f006f6c0415c8853e55b56a2ed14019c5165cd4278: Status 404 returned error can't find the container with id f377dc8c3a3de3bc602471f006f6c0415c8853e55b56a2ed14019c5165cd4278 Jun 11 15:00:04.927483 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:04.927446 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6nv4q" event={"ID":"595146ac-1555-4472-8500-1f4b84565a05","Type":"ContainerStarted","Data":"f377dc8c3a3de3bc602471f006f6c0415c8853e55b56a2ed14019c5165cd4278"} Jun 11 15:00:05.931706 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:05.931674 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6nv4q" event={"ID":"595146ac-1555-4472-8500-1f4b84565a05","Type":"ContainerStarted","Data":"423c45497860246efd8e789286b2844d942c8d54ad9f82f73529ca985d9379ff"} Jun 11 15:00:05.932066 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:05.931740 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6nv4q" event={"ID":"595146ac-1555-4472-8500-1f4b84565a05","Type":"ContainerStarted","Data":"db0fbd3b3b04ebba00a5542058544b6b083afd020ac1de1401be7c86693e797f"} Jun 11 15:00:07.937764 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:07.937731 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6nv4q" event={"ID":"595146ac-1555-4472-8500-1f4b84565a05","Type":"ContainerStarted","Data":"3fd1e5b2ed773e3f62362bc7b753c5c86ca215d53c33f7d4e192a1f3eea926ce"} Jun 11 15:00:07.958615 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:07.958573 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6nv4q" podStartSLOduration=17.723548589 podStartE2EDuration="19.958558834s" totalImagesPullingTime="2.235010245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:07.957054312 +0000 UTC m=+64.733686043" watchObservedRunningTime="2026-06-11 15:00:07.958558834 +0000 UTC m=+64.735190555" Jun 11 15:00:09.446379 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.446331 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 15:00:09.446815 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.446439 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 15:00:09.446815 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.446509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 15:00:09.449328 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.449309 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 15:00:09.450454 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.450441 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 15:00:09.450495 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.450445 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 15:00:09.458690 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.458650 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46e97f49-b264-4a5e-8f66-9ae0ea812103-metrics-certs\") pod \"network-metrics-daemon-jxjxg\" (UID: \"46e97f49-b264-4a5e-8f66-9ae0ea812103\") " pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 15:00:09.458690 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.458666 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b6572eff-fb22-4baa-b130-995f9284a608-original-pull-secret\") pod \"global-pull-secret-syncer-2zl8k\" (UID: \"b6572eff-fb22-4baa-b130-995f9284a608\") " pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 15:00:09.459313 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.459297 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 15:00:09.469385 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.469366 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45rd\" (UniqueName: \"kubernetes.io/projected/159b39a6-357d-4362-800e-2bb1d2fca812-kube-api-access-c45rd\") pod \"network-check-target-5277h\" (UID: \"159b39a6-357d-4362-800e-2bb1d2fca812\") " pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 15:00:09.578772 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.578732 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zl8k" Jun 11 15:00:09.586434 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.586411 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p4p2b\"" Jun 11 15:00:09.590663 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.590644 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdl6h\"" Jun 11 15:00:09.593779 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.593761 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jxjxg" Jun 11 15:00:09.599461 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.599440 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 15:00:09.717243 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.717153 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2zl8k"] Jun 11 15:00:09.720934 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:09.720909 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6572eff_fb22_4baa_b130_995f9284a608.slice/crio-7eb4d5a6040e5828e694e637b59b77508046484b7c95db929d1c3b194d3a9d55 WatchSource:0}: Error finding container 7eb4d5a6040e5828e694e637b59b77508046484b7c95db929d1c3b194d3a9d55: Status 404 returned error can't find the container with id 7eb4d5a6040e5828e694e637b59b77508046484b7c95db929d1c3b194d3a9d55 Jun 11 15:00:09.730569 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.730545 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jxjxg"] Jun 11 15:00:09.734374 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:09.734347 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e97f49_b264_4a5e_8f66_9ae0ea812103.slice/crio-6c481f749031efe68edb5f025677b1fa51db4ec6017a7f4d1558e020d9a73821 WatchSource:0}: Error finding container 6c481f749031efe68edb5f025677b1fa51db4ec6017a7f4d1558e020d9a73821: Status 404 returned error can't find the container with id 6c481f749031efe68edb5f025677b1fa51db4ec6017a7f4d1558e020d9a73821 Jun 11 15:00:09.748805 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.748757 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5277h"] Jun 11 15:00:09.751083 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:09.751063 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159b39a6_357d_4362_800e_2bb1d2fca812.slice/crio-7599711b395c8298d649073d2efe623462be3c6e7969103dd978d89089bb1d71 WatchSource:0}: Error finding container 7599711b395c8298d649073d2efe623462be3c6e7969103dd978d89089bb1d71: Status 404 returned error can't find the container with id 7599711b395c8298d649073d2efe623462be3c6e7969103dd978d89089bb1d71 Jun 11 15:00:09.944410 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.944379 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jxjxg" event={"ID":"46e97f49-b264-4a5e-8f66-9ae0ea812103","Type":"ContainerStarted","Data":"6c481f749031efe68edb5f025677b1fa51db4ec6017a7f4d1558e020d9a73821"} Jun 11 15:00:09.945435 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.945410 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2zl8k" event={"ID":"b6572eff-fb22-4baa-b130-995f9284a608","Type":"ContainerStarted","Data":"7eb4d5a6040e5828e694e637b59b77508046484b7c95db929d1c3b194d3a9d55"} Jun 11 15:00:09.946368 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:09.946348 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5277h" event={"ID":"159b39a6-357d-4362-800e-2bb1d2fca812","Type":"ContainerStarted","Data":"7599711b395c8298d649073d2efe623462be3c6e7969103dd978d89089bb1d71"} Jun 11 15:00:10.224591 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.224557 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn"] Jun 11 15:00:10.226956 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.226933 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.230978 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.230849 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 11 15:00:10.230978 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.230861 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 11 15:00:10.232045 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.232023 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 11 15:00:10.232045 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.232036 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 11 15:00:10.232298 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.232104 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 11 15:00:10.232298 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.232085 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 11 15:00:10.232634 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.232616 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 11 15:00:10.242359 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.242336 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn"] Jun 11 15:00:10.283700 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.283669 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58696fc5c4-rjdjt"] Jun 11 15:00:10.283952 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:00:10.283929 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" podUID="b8f84c27-943e-4dd0-83e0-4f24a95114bf" Jun 11 15:00:10.354072 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.354039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.354231 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.354085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-hub\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.354231 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.354208 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74924442-056e-425b-b5ab-2159f90399f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.354363 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.354267 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.354363 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.354320 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw5n5\" (UniqueName: \"kubernetes.io/projected/74924442-056e-425b-b5ab-2159f90399f4-kube-api-access-sw5n5\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.354523 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.354426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-ca\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.456790 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.455586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-ca\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.456790 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.455636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.456790 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.455663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-hub\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.456790 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.455737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74924442-056e-425b-b5ab-2159f90399f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.456790 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.455780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.456790 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.455805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw5n5\" (UniqueName: \"kubernetes.io/projected/74924442-056e-425b-b5ab-2159f90399f4-kube-api-access-sw5n5\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.457828 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.457777 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74924442-056e-425b-b5ab-2159f90399f4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.461249 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.461204 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-ca\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.464248 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.464194 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-hub\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.464402 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.464338 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.465106 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.465059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74924442-056e-425b-b5ab-2159f90399f4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.468298 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.468274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw5n5\" (UniqueName: \"kubernetes.io/projected/74924442-056e-425b-b5ab-2159f90399f4-kube-api-access-sw5n5\") pod \"cluster-proxy-proxy-agent-9bf94d4b4-5mnxn\" (UID: \"74924442-056e-425b-b5ab-2159f90399f4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.546844 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.546762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" Jun 11 15:00:10.557237 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.557212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 15:00:10.560060 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.560038 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/94eacb66-220d-4890-90a3-5840d70579ae-metrics-tls\") pod \"dns-default-zvgk5\" (UID: \"94eacb66-220d-4890-90a3-5840d70579ae\") " pod="openshift-dns/dns-default-zvgk5" Jun 11 15:00:10.749784 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.749749 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmhvb\"" Jun 11 15:00:10.757428 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.757190 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvgk5" Jun 11 15:00:10.949456 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.949434 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 15:00:10.956740 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.956585 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 15:00:10.961359 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.961077 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn"] Jun 11 15:00:10.977263 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:10.977180 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvgk5"] Jun 11 15:00:10.981346 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:10.981320 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94eacb66_220d_4890_90a3_5840d70579ae.slice/crio-3349068c0f69e721f5e8eeae7829c18f17e88e88334da9b3ef7dbc0cd16375e5 WatchSource:0}: Error finding container 3349068c0f69e721f5e8eeae7829c18f17e88e88334da9b3ef7dbc0cd16375e5: Status 404 returned error can't find the container with id 3349068c0f69e721f5e8eeae7829c18f17e88e88334da9b3ef7dbc0cd16375e5 Jun 11 15:00:11.063527 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.063429 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-certificates\") pod \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " Jun 11 15:00:11.063527 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.063486 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8f84c27-943e-4dd0-83e0-4f24a95114bf-ca-trust-extracted\") pod \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " Jun 11 15:00:11.063652 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.063529 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-bound-sa-token\") pod \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " Jun 11 15:00:11.063652 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.063590 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9rb2\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-kube-api-access-z9rb2\") pod \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " Jun 11 15:00:11.063652 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.063642 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-installation-pull-secrets\") pod \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " Jun 11 15:00:11.063820 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.063670 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-image-registry-private-configuration\") pod \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " Jun 11 15:00:11.063820 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.063732 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-trusted-ca\") pod \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\" (UID: \"b8f84c27-943e-4dd0-83e0-4f24a95114bf\") " Jun 11 15:00:11.064094 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.064059 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b8f84c27-943e-4dd0-83e0-4f24a95114bf" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:00:11.064457 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.064430 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8f84c27-943e-4dd0-83e0-4f24a95114bf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b8f84c27-943e-4dd0-83e0-4f24a95114bf" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 15:00:11.064969 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.064926 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b8f84c27-943e-4dd0-83e0-4f24a95114bf" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:00:11.066043 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.066005 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b8f84c27-943e-4dd0-83e0-4f24a95114bf" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:00:11.066322 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.066258 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b8f84c27-943e-4dd0-83e0-4f24a95114bf" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:00:11.066757 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.066708 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b8f84c27-943e-4dd0-83e0-4f24a95114bf" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:00:11.067985 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.067948 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-kube-api-access-z9rb2" (OuterVolumeSpecName: "kube-api-access-z9rb2") pod "b8f84c27-943e-4dd0-83e0-4f24a95114bf" (UID: "b8f84c27-943e-4dd0-83e0-4f24a95114bf"). InnerVolumeSpecName "kube-api-access-z9rb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:00:11.164994 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.164949 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9rb2\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-kube-api-access-z9rb2\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:00:11.164994 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.164982 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-installation-pull-secrets\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:00:11.164994 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.164996 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b8f84c27-943e-4dd0-83e0-4f24a95114bf-image-registry-private-configuration\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:00:11.165227 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.165009 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-trusted-ca\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:00:11.165227 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.165021 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-certificates\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:00:11.165227 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.165032 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b8f84c27-943e-4dd0-83e0-4f24a95114bf-ca-trust-extracted\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:00:11.165227 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.165043 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-bound-sa-token\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:00:11.958023 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.957508 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jxjxg" event={"ID":"46e97f49-b264-4a5e-8f66-9ae0ea812103","Type":"ContainerStarted","Data":"c85ad265077fcbc411db2707c0980b1718e3c007c762197d42250c7df2bb3747"} Jun 11 15:00:11.958023 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.957574 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jxjxg" event={"ID":"46e97f49-b264-4a5e-8f66-9ae0ea812103","Type":"ContainerStarted","Data":"8ae1f76d35ea13aa736ad32ec068195f7a77fc5d71a3b733ede6e4f0d912d6c2"} Jun 11 15:00:11.959406 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.959377 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvgk5" event={"ID":"94eacb66-220d-4890-90a3-5840d70579ae","Type":"ContainerStarted","Data":"3349068c0f69e721f5e8eeae7829c18f17e88e88334da9b3ef7dbc0cd16375e5"} Jun 11 15:00:11.961510 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.961486 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58696fc5c4-rjdjt" Jun 11 15:00:11.961975 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.961953 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" event={"ID":"74924442-056e-425b-b5ab-2159f90399f4","Type":"ContainerStarted","Data":"cb4e6685d6e40a64b50472379b4e48bdcf5eaecc8c909eb6e12f780e41024175"} Jun 11 15:00:11.978416 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:11.978041 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jxjxg" podStartSLOduration=67.895443098 podStartE2EDuration="1m8.978021388s" totalImagesPullingTime="1.08257829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:11.975935169 +0000 UTC m=+68.752566892" watchObservedRunningTime="2026-06-11 15:00:11.978021388 +0000 UTC m=+68.754653110" Jun 11 15:00:12.015275 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.015243 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58696fc5c4-rjdjt"] Jun 11 15:00:12.021970 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.021911 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-58696fc5c4-rjdjt"] Jun 11 15:00:12.080153 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.080067 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b8f84c27-943e-4dd0-83e0-4f24a95114bf-registry-tls\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:00:12.887676 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.887533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 15:00:12.890201 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.890178 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/848c9644-faa6-430e-8cc1-b8b3f451f7bd-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-wq5z5\" (UID: \"848c9644-faa6-430e-8cc1-b8b3f451f7bd\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 15:00:12.988694 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.988659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:12.989107 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.988735 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:12.989456 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.989428 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c39500a-f1fc-4a5f-b922-1ff7217234b0-service-ca-bundle\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:12.991179 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:12.991153 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c39500a-f1fc-4a5f-b922-1ff7217234b0-metrics-certs\") pod \"router-default-868f598b87-7svzh\" (UID: \"5c39500a-f1fc-4a5f-b922-1ff7217234b0\") " pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:13.092384 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:13.092349 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7rxgc\"" Jun 11 15:00:13.099851 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:13.099828 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" Jun 11 15:00:13.224215 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:13.224139 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rnwdz\"" Jun 11 15:00:13.231700 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:13.231677 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:13.673973 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:13.673940 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8f84c27-943e-4dd0-83e0-4f24a95114bf" path="/var/lib/kubelet/pods/b8f84c27-943e-4dd0-83e0-4f24a95114bf/volumes" Jun 11 15:00:14.974966 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:14.974937 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5277h" event={"ID":"159b39a6-357d-4362-800e-2bb1d2fca812","Type":"ContainerStarted","Data":"bfb1fc9c59076b7025f8a45d80c67c32cc32642d776715b54c255e7d6a9f3747"} Jun 11 15:00:14.975373 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:14.975069 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 15:00:14.989384 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:14.989356 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-868f598b87-7svzh"] Jun 11 15:00:14.991780 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:14.991752 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c39500a_f1fc_4a5f_b922_1ff7217234b0.slice/crio-3c46e7466a513445a60fea952fcee3bcf526d1cd76e7491883bf46137bd96c97 WatchSource:0}: Error finding container 3c46e7466a513445a60fea952fcee3bcf526d1cd76e7491883bf46137bd96c97: Status 404 returned error can't find the container with id 3c46e7466a513445a60fea952fcee3bcf526d1cd76e7491883bf46137bd96c97 Jun 11 15:00:14.995013 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:14.994963 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5277h" podStartSLOduration=66.916861767 podStartE2EDuration="1m11.994945213s" totalImagesPullingTime="5.078083446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:14.993706329 +0000 UTC m=+71.770338069" watchObservedRunningTime="2026-06-11 15:00:14.994945213 +0000 UTC m=+71.771576950" Jun 11 15:00:15.010518 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.008657 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5"] Jun 11 15:00:15.015620 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:15.015595 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848c9644_faa6_430e_8cc1_b8b3f451f7bd.slice/crio-a4f1e8abee5ef63892b5f11668ce81b021a19d8dcbd25364613a6a96e7831319 WatchSource:0}: Error finding container a4f1e8abee5ef63892b5f11668ce81b021a19d8dcbd25364613a6a96e7831319: Status 404 returned error can't find the container with id a4f1e8abee5ef63892b5f11668ce81b021a19d8dcbd25364613a6a96e7831319 Jun 11 15:00:15.980870 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.980836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2zl8k" event={"ID":"b6572eff-fb22-4baa-b130-995f9284a608","Type":"ContainerStarted","Data":"9f0af42e5093723f5402ae9394cd8497baddaf4fc028cc92563fa15e15666724"} Jun 11 15:00:15.982530 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.982506 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvgk5" event={"ID":"94eacb66-220d-4890-90a3-5840d70579ae","Type":"ContainerStarted","Data":"6287944ef29e54f732eb7a2892a61726384da7344868ed679f1c5c2afcd56169"} Jun 11 15:00:15.982638 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.982551 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zvgk5" Jun 11 15:00:15.982638 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.982567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvgk5" event={"ID":"94eacb66-220d-4890-90a3-5840d70579ae","Type":"ContainerStarted","Data":"545d3319f39eaba2665a9e0b4cf9a24fffcb0fbe437e23e23d3d1e1f5c4ecc16"} Jun 11 15:00:15.984044 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.984013 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-868f598b87-7svzh" event={"ID":"5c39500a-f1fc-4a5f-b922-1ff7217234b0","Type":"ContainerStarted","Data":"83423a05b350c949f0eadeb157f379fdf7b2feccc1a06038daed67018f32fb82"} Jun 11 15:00:15.984151 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.984061 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-868f598b87-7svzh" event={"ID":"5c39500a-f1fc-4a5f-b922-1ff7217234b0","Type":"ContainerStarted","Data":"3c46e7466a513445a60fea952fcee3bcf526d1cd76e7491883bf46137bd96c97"} Jun 11 15:00:15.985237 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.985218 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" event={"ID":"848c9644-faa6-430e-8cc1-b8b3f451f7bd","Type":"ContainerStarted","Data":"a4f1e8abee5ef63892b5f11668ce81b021a19d8dcbd25364613a6a96e7831319"} Jun 11 15:00:15.999699 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:15.999656 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2zl8k" podStartSLOduration=67.870706107 podStartE2EDuration="1m12.999644807s" totalImagesPullingTime="5.1289387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:15.999018732 +0000 UTC m=+72.775650452" watchObservedRunningTime="2026-06-11 15:00:15.999644807 +0000 UTC m=+72.776276527" Jun 11 15:00:16.023819 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:16.023782 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-868f598b87-7svzh" podStartSLOduration=35.02377105 podStartE2EDuration="35.02377105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 14:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:16.023294381 +0000 UTC m=+72.799926112" watchObservedRunningTime="2026-06-11 15:00:16.02377105 +0000 UTC m=+72.800402748" Jun 11 15:00:16.047065 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:16.047010 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zvgk5" podStartSLOduration=34.195874849 podStartE2EDuration="38.046991778s" totalImagesPullingTime="3.851116929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 14:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:16.045910084 +0000 UTC m=+72.822541810" watchObservedRunningTime="2026-06-11 15:00:16.046991778 +0000 UTC m=+72.823623499" Jun 11 15:00:16.231940 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:16.231843 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:16.235079 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:16.235058 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:16.988776 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:16.988755 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:16.990239 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:16.990219 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-868f598b87-7svzh" Jun 11 15:00:17.992414 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:17.992382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" event={"ID":"848c9644-faa6-430e-8cc1-b8b3f451f7bd","Type":"ContainerStarted","Data":"e8e700d6ded7630bfe3854790a64761857d6aa2bb2154399075a13e25948473f"} Jun 11 15:00:18.014572 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:18.014524 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-wq5z5" podStartSLOduration=36.057516275 podStartE2EDuration="38.014510271s" totalImagesPullingTime="1.956993996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 14:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:18.014042727 +0000 UTC m=+74.790674458" watchObservedRunningTime="2026-06-11 15:00:18.014510271 +0000 UTC m=+74.791141994" Jun 11 15:00:25.187266 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.187235 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz"] Jun 11 15:00:25.189772 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.189752 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.193254 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.193234 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 11 15:00:25.193485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.193469 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 11 15:00:25.194448 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.194434 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 15:00:25.194890 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.194861 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 11 15:00:25.194998 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.194902 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-szwqt\"" Jun 11 15:00:25.219339 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.219311 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dvjtj"] Jun 11 15:00:25.221584 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.221567 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz"] Jun 11 15:00:25.221679 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.221656 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.224408 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.224390 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 15:00:25.224687 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.224673 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p9wlr\"" Jun 11 15:00:25.225082 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.225058 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 15:00:25.225189 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.225174 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 15:00:25.280838 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.280806 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.280838 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.280842 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndwkx\" (UniqueName: \"kubernetes.io/projected/73a6403f-a8fc-4803-867c-501b5b67eec8-kube-api-access-ndwkx\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.281020 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.280860 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.281020 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.280878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.281020 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.280903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-root\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.281020 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.280956 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d640b8eb-8644-42cc-9303-d8af2af0457d-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.281020 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.280980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-wtmp\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.281189 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.281017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d640b8eb-8644-42cc-9303-d8af2af0457d-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.281189 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.281082 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8w9v\" (UniqueName: \"kubernetes.io/projected/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-api-access-v8w9v\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.281189 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.281103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-tls\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.281189 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.281119 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-textfile\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.281189 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.281139 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-sys\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.281189 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.281170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a6403f-a8fc-4803-867c-501b5b67eec8-metrics-client-ca\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.281383 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.281243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.281383 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.281264 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.381980 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.381943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-tls\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.381980 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.381981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-textfile\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-sys\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382030 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a6403f-a8fc-4803-867c-501b5b67eec8-metrics-client-ca\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382077 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-sys\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382092 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:00:25.382105 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:00:25.382180 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-tls podName:73a6403f-a8fc-4803-867c-501b5b67eec8 nodeName:}" failed. No retries permitted until 2026-06-11 15:00:25.88215873 +0000 UTC m=+82.658790430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-tls") pod "node-exporter-dvjtj" (UID: "73a6403f-a8fc-4803-867c-501b5b67eec8") : secret "node-exporter-tls" not found Jun 11 15:00:25.382218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndwkx\" (UniqueName: \"kubernetes.io/projected/73a6403f-a8fc-4803-867c-501b5b67eec8-kube-api-access-ndwkx\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382243 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.382666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.382666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382300 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-root\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d640b8eb-8644-42cc-9303-d8af2af0457d-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.382666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382358 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-textfile\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-wtmp\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.382666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d640b8eb-8644-42cc-9303-d8af2af0457d-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.382666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8w9v\" (UniqueName: \"kubernetes.io/projected/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-api-access-v8w9v\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.383078 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382704 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a6403f-a8fc-4803-867c-501b5b67eec8-metrics-client-ca\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.383078 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382781 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.383078 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382835 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d640b8eb-8644-42cc-9303-d8af2af0457d-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.383078 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-root\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.383078 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382853 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.383078 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.382840 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-wtmp\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.383453 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.383429 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d640b8eb-8644-42cc-9303-d8af2af0457d-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.384650 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.384624 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.384867 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.384850 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.385059 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.385040 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.395939 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.395917 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndwkx\" (UniqueName: \"kubernetes.io/projected/73a6403f-a8fc-4803-867c-501b5b67eec8-kube-api-access-ndwkx\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.399035 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.399017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8w9v\" (UniqueName: \"kubernetes.io/projected/d640b8eb-8644-42cc-9303-d8af2af0457d-kube-api-access-v8w9v\") pod \"kube-state-metrics-57bbf8bfb5-hctsz\" (UID: \"d640b8eb-8644-42cc-9303-d8af2af0457d\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.498083 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.498012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" Jun 11 15:00:25.637824 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.637795 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz"] Jun 11 15:00:25.640778 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:25.640755 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd640b8eb_8644_42cc_9303_d8af2af0457d.slice/crio-ebd82966cbdee7eadb54b55e55067e8a7bafca6e58456fa2f86e2239d6194985 WatchSource:0}: Error finding container ebd82966cbdee7eadb54b55e55067e8a7bafca6e58456fa2f86e2239d6194985: Status 404 returned error can't find the container with id ebd82966cbdee7eadb54b55e55067e8a7bafca6e58456fa2f86e2239d6194985 Jun 11 15:00:25.885605 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.885572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-tls\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.887892 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.887873 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73a6403f-a8fc-4803-867c-501b5b67eec8-node-exporter-tls\") pod \"node-exporter-dvjtj\" (UID: \"73a6403f-a8fc-4803-867c-501b5b67eec8\") " pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:25.991061 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:25.991034 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zvgk5" Jun 11 15:00:26.027336 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:26.027309 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" event={"ID":"d640b8eb-8644-42cc-9303-d8af2af0457d","Type":"ContainerStarted","Data":"ebd82966cbdee7eadb54b55e55067e8a7bafca6e58456fa2f86e2239d6194985"} Jun 11 15:00:26.130776 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:26.130738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dvjtj" Jun 11 15:00:26.141890 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:26.141866 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a6403f_a8fc_4803_867c_501b5b67eec8.slice/crio-202faecf611a242cc8165ee9623d5b3b1609f251945b3f419017af660e0dbbb6 WatchSource:0}: Error finding container 202faecf611a242cc8165ee9623d5b3b1609f251945b3f419017af660e0dbbb6: Status 404 returned error can't find the container with id 202faecf611a242cc8165ee9623d5b3b1609f251945b3f419017af660e0dbbb6 Jun 11 15:00:27.032450 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:27.032409 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dvjtj" event={"ID":"73a6403f-a8fc-4803-867c-501b5b67eec8","Type":"ContainerStarted","Data":"202faecf611a242cc8165ee9623d5b3b1609f251945b3f419017af660e0dbbb6"} Jun 11 15:00:28.041749 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.041638 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" event={"ID":"d640b8eb-8644-42cc-9303-d8af2af0457d","Type":"ContainerStarted","Data":"0974af2100e860ce63b2a4a1dd44d44bb82648904f8940063595ddbc596a60ef"} Jun 11 15:00:28.043703 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.043649 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" event={"ID":"74924442-056e-425b-b5ab-2159f90399f4","Type":"ContainerStarted","Data":"6ffd90d8042c019b6ab72be6db57aa1cb22f0d0e62e4c3faaa3e0f1a1b7efd47"} Jun 11 15:00:28.313240 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.313206 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6b948c97b5-gp2m9"] Jun 11 15:00:28.316757 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.316736 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.320015 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.319702 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 11 15:00:28.320110 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.320023 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f6eqloqmk1dgg\"" Jun 11 15:00:28.320156 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.319927 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 11 15:00:28.320210 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.319923 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 11 15:00:28.320262 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.319976 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j4f8n\"" Jun 11 15:00:28.320318 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.319965 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 11 15:00:28.320318 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.319980 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 11 15:00:28.335318 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.335237 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b948c97b5-gp2m9"] Jun 11 15:00:28.407361 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.407287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e50e8c30-4f56-4f29-9332-74049e07838c-metrics-client-ca\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.407361 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.407340 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.407569 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.407371 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.407569 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.407443 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-tls\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.407569 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.407511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzgs2\" (UniqueName: \"kubernetes.io/projected/e50e8c30-4f56-4f29-9332-74049e07838c-kube-api-access-lzgs2\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.407689 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.407609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.407689 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.407637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-grpc-tls\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.407689 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.407667 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.508445 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.508422 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.508559 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.508454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-grpc-tls\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.508559 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.508477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.508559 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.508511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e50e8c30-4f56-4f29-9332-74049e07838c-metrics-client-ca\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.508559 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.508546 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.508864 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.508576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.508864 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.508607 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-tls\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.508864 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.508637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzgs2\" (UniqueName: \"kubernetes.io/projected/e50e8c30-4f56-4f29-9332-74049e07838c-kube-api-access-lzgs2\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.510436 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.510409 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e50e8c30-4f56-4f29-9332-74049e07838c-metrics-client-ca\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.511626 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.511600 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-grpc-tls\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.511738 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.511606 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.511810 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.511779 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.511979 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.511961 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.512473 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.512456 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-tls\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.513417 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.513398 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e50e8c30-4f56-4f29-9332-74049e07838c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.522582 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.522557 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzgs2\" (UniqueName: \"kubernetes.io/projected/e50e8c30-4f56-4f29-9332-74049e07838c-kube-api-access-lzgs2\") pod \"thanos-querier-6b948c97b5-gp2m9\" (UID: \"e50e8c30-4f56-4f29-9332-74049e07838c\") " pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.627504 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.627468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:28.771782 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:28.771750 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6b948c97b5-gp2m9"] Jun 11 15:00:28.775098 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:28.775045 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50e8c30_4f56_4f29_9332_74049e07838c.slice/crio-75e147bcebe486835a7c60ed6766a43aea17d8338c6284fcd99430d264ff20e3 WatchSource:0}: Error finding container 75e147bcebe486835a7c60ed6766a43aea17d8338c6284fcd99430d264ff20e3: Status 404 returned error can't find the container with id 75e147bcebe486835a7c60ed6766a43aea17d8338c6284fcd99430d264ff20e3 Jun 11 15:00:29.049405 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.049321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" event={"ID":"d640b8eb-8644-42cc-9303-d8af2af0457d","Type":"ContainerStarted","Data":"2fad3da1fb0a89c9f4da39b47c97589e91121e2174c43f0a44100e894817d323"} Jun 11 15:00:29.049405 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.049376 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" event={"ID":"d640b8eb-8644-42cc-9303-d8af2af0457d","Type":"ContainerStarted","Data":"8720f58bde664a8c91f083ca160e1b2a7a2966481a4b228b086f205addd21804"} Jun 11 15:00:29.050557 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.050532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" event={"ID":"e50e8c30-4f56-4f29-9332-74049e07838c","Type":"ContainerStarted","Data":"75e147bcebe486835a7c60ed6766a43aea17d8338c6284fcd99430d264ff20e3"} Jun 11 15:00:29.052587 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.052563 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" event={"ID":"74924442-056e-425b-b5ab-2159f90399f4","Type":"ContainerStarted","Data":"d73e7d993935a6ed733ae2c6fd4b46aa5fd7e288e9a8dd9b9c6099daa2fce193"} Jun 11 15:00:29.052704 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.052599 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" event={"ID":"74924442-056e-425b-b5ab-2159f90399f4","Type":"ContainerStarted","Data":"217b4fd67252b6fd6f6e30fb91b0a8e105c7dfda157f0adf48f35ab85d7faddf"} Jun 11 15:00:29.054123 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.054103 2562 generic.go:356] "Generic (PLEG): container finished" podID="73a6403f-a8fc-4803-867c-501b5b67eec8" containerID="ca3aca71c37e95bf04e8a181e3b44b23c41e0bb182f671297d3244896dd8a6ca" exitCode=0 Jun 11 15:00:29.054216 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.054137 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dvjtj" event={"ID":"73a6403f-a8fc-4803-867c-501b5b67eec8","Type":"ContainerDied","Data":"ca3aca71c37e95bf04e8a181e3b44b23c41e0bb182f671297d3244896dd8a6ca"} Jun 11 15:00:29.079844 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.079790 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-hctsz" podStartSLOduration=1.826822269 podStartE2EDuration="4.079771264s" totalImagesPullingTime="2.252948995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:29.079185969 +0000 UTC m=+85.855817704" watchObservedRunningTime="2026-06-11 15:00:29.079771264 +0000 UTC m=+85.856402987" Jun 11 15:00:29.134789 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:29.134709 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-9bf94d4b4-5mnxn" podStartSLOduration=2.186210012 podStartE2EDuration="19.134687873s" totalImagesPullingTime="16.948477861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:29.13333652 +0000 UTC m=+85.909968241" watchObservedRunningTime="2026-06-11 15:00:29.134687873 +0000 UTC m=+85.911319597" Jun 11 15:00:30.060911 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:30.060878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dvjtj" event={"ID":"73a6403f-a8fc-4803-867c-501b5b67eec8","Type":"ContainerStarted","Data":"907a43abe8a6b50dd2e9a66eb8bd7b5616f598ce0785f9a7b719aff5bc337b47"} Jun 11 15:00:30.061363 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:30.060937 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dvjtj" event={"ID":"73a6403f-a8fc-4803-867c-501b5b67eec8","Type":"ContainerStarted","Data":"46a3a01d2ce8c3ee81dab777f103042aaabe250affd030bc67ceba5cbfa209e0"} Jun 11 15:00:32.072131 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:32.072093 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" event={"ID":"e50e8c30-4f56-4f29-9332-74049e07838c","Type":"ContainerStarted","Data":"5f48e2ff181cb17de770422ce0b6b6f6d602d6da7a03b6992b4f7fe1f5d928e1"} Jun 11 15:00:32.072484 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:32.072161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" event={"ID":"e50e8c30-4f56-4f29-9332-74049e07838c","Type":"ContainerStarted","Data":"b2ebcab6b9f41b03d0022c79a27c8ded6e439e89ea1023bfa4146d9c75080a9a"} Jun 11 15:00:32.072484 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:32.072187 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" event={"ID":"e50e8c30-4f56-4f29-9332-74049e07838c","Type":"ContainerStarted","Data":"f40697b7d01277a344d0c9989970972447f20b56e16ded5c3f1ba579796b696a"} Jun 11 15:00:33.078189 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:33.078150 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" event={"ID":"e50e8c30-4f56-4f29-9332-74049e07838c","Type":"ContainerStarted","Data":"3b913d505163482a94994466ef58136eb8f7acad65b428fb1c6e3982b50e127b"} Jun 11 15:00:33.078574 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:33.078227 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" event={"ID":"e50e8c30-4f56-4f29-9332-74049e07838c","Type":"ContainerStarted","Data":"715fb38f87c9f80f3ade0443a1736098a00e71511d3e98d2575231f123ec3a6d"} Jun 11 15:00:33.078574 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:33.078268 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" event={"ID":"e50e8c30-4f56-4f29-9332-74049e07838c","Type":"ContainerStarted","Data":"d4e719093967e2dea8c9813154f285d0fa3882eefae5b6e1d705e17d21b952e8"} Jun 11 15:00:33.078574 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:33.078420 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:33.107682 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:33.107629 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dvjtj" podStartSLOduration=5.682092469 podStartE2EDuration="8.107610237s" totalImagesPullingTime="2.357980109s" totalInitContainerRuntime="67.537659ms" isStatefulPod=true podCreationTimestamp="2026-06-11 15:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:30.085640838 +0000 UTC m=+86.862272558" watchObservedRunningTime="2026-06-11 15:00:33.107610237 +0000 UTC m=+89.884241989" Jun 11 15:00:33.108077 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:33.108033 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" podStartSLOduration=1.730134888 podStartE2EDuration="5.108024858s" totalImagesPullingTime="3.37788997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:00:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:33.107455017 +0000 UTC m=+89.884086751" watchObservedRunningTime="2026-06-11 15:00:33.108024858 +0000 UTC m=+89.884656579" Jun 11 15:00:34.636960 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.636930 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-g2qng"] Jun 11 15:00:34.639335 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.639315 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-g2qng" Jun 11 15:00:34.642588 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.642565 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vnvhr\"" Jun 11 15:00:34.642753 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.642735 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 15:00:34.642979 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.642965 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 15:00:34.652598 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.652578 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-g2qng"] Jun 11 15:00:34.764276 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.764239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkrvj\" (UniqueName: \"kubernetes.io/projected/0b2a36ef-be5f-4768-8925-891462a9549a-kube-api-access-wkrvj\") pod \"downloads-6657875f97-g2qng\" (UID: \"0b2a36ef-be5f-4768-8925-891462a9549a\") " pod="openshift-console/downloads-6657875f97-g2qng" Jun 11 15:00:34.865097 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.865061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkrvj\" (UniqueName: \"kubernetes.io/projected/0b2a36ef-be5f-4768-8925-891462a9549a-kube-api-access-wkrvj\") pod \"downloads-6657875f97-g2qng\" (UID: \"0b2a36ef-be5f-4768-8925-891462a9549a\") " pod="openshift-console/downloads-6657875f97-g2qng" Jun 11 15:00:34.875079 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.875054 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkrvj\" (UniqueName: \"kubernetes.io/projected/0b2a36ef-be5f-4768-8925-891462a9549a-kube-api-access-wkrvj\") pod \"downloads-6657875f97-g2qng\" (UID: \"0b2a36ef-be5f-4768-8925-891462a9549a\") " pod="openshift-console/downloads-6657875f97-g2qng" Jun 11 15:00:34.949073 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:34.948997 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-g2qng" Jun 11 15:00:35.075183 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:35.075151 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-g2qng"] Jun 11 15:00:35.078000 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:35.077973 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b2a36ef_be5f_4768_8925_891462a9549a.slice/crio-91dfbee3d5229be49ed3ad9f1dbabf1a587ceb21fe581ecd241d93f79ab4f61c WatchSource:0}: Error finding container 91dfbee3d5229be49ed3ad9f1dbabf1a587ceb21fe581ecd241d93f79ab4f61c: Status 404 returned error can't find the container with id 91dfbee3d5229be49ed3ad9f1dbabf1a587ceb21fe581ecd241d93f79ab4f61c Jun 11 15:00:35.086733 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:35.086690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-g2qng" event={"ID":"0b2a36ef-be5f-4768-8925-891462a9549a","Type":"ContainerStarted","Data":"91dfbee3d5229be49ed3ad9f1dbabf1a587ceb21fe581ecd241d93f79ab4f61c"} Jun 11 15:00:39.088347 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:39.088318 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6b948c97b5-gp2m9" Jun 11 15:00:41.370815 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.370776 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56687ff765-cdk8m"] Jun 11 15:00:41.374082 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.374058 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.376969 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.376929 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 15:00:41.377065 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.376982 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 15:00:41.377065 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.377054 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 15:00:41.377175 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.377142 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 15:00:41.378555 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.378398 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f29lk\"" Jun 11 15:00:41.378555 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.378498 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 15:00:41.391880 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.391853 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56687ff765-cdk8m"] Jun 11 15:00:41.523585 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.523541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-config\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.523791 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.523599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-serving-cert\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.523791 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.523629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-service-ca\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.523791 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.523661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pktx9\" (UniqueName: \"kubernetes.io/projected/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-kube-api-access-pktx9\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.523791 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.523685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-oauth-serving-cert\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.524009 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.523824 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-oauth-config\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625112 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625030 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-oauth-config\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625112 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-config\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625112 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-serving-cert\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625112 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625113 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-service-ca\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625413 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625237 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pktx9\" (UniqueName: \"kubernetes.io/projected/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-kube-api-access-pktx9\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625413 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-oauth-serving-cert\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625763 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625737 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-service-ca\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625899 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625858 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-config\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.625961 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.625946 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-oauth-serving-cert\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.627507 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.627479 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-oauth-config\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.627617 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.627522 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-serving-cert\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.635675 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.635652 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pktx9\" (UniqueName: \"kubernetes.io/projected/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-kube-api-access-pktx9\") pod \"console-56687ff765-cdk8m\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.684745 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.684697 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:41.831780 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:41.831651 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56687ff765-cdk8m"] Jun 11 15:00:41.834410 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:00:41.834380 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a8a9db5_dccd_4a3e_94b7_c823c61a4132.slice/crio-c5d0927746c3b0bb3d5b7debf835bb3821fbefecec0e1c81801d9f6d15231dd2 WatchSource:0}: Error finding container c5d0927746c3b0bb3d5b7debf835bb3821fbefecec0e1c81801d9f6d15231dd2: Status 404 returned error can't find the container with id c5d0927746c3b0bb3d5b7debf835bb3821fbefecec0e1c81801d9f6d15231dd2 Jun 11 15:00:42.117523 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:42.117487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56687ff765-cdk8m" event={"ID":"7a8a9db5-dccd-4a3e-94b7-c823c61a4132","Type":"ContainerStarted","Data":"c5d0927746c3b0bb3d5b7debf835bb3821fbefecec0e1c81801d9f6d15231dd2"} Jun 11 15:00:45.131597 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:45.131555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56687ff765-cdk8m" event={"ID":"7a8a9db5-dccd-4a3e-94b7-c823c61a4132","Type":"ContainerStarted","Data":"c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c"} Jun 11 15:00:45.150696 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:45.150637 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56687ff765-cdk8m" podStartSLOduration=0.984898402 podStartE2EDuration="4.150616738s" totalImagesPullingTime="3.165718336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:45.149255984 +0000 UTC m=+101.925887729" watchObservedRunningTime="2026-06-11 15:00:45.150616738 +0000 UTC m=+101.927248461" Jun 11 15:00:45.987651 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:45.987612 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5277h" Jun 11 15:00:51.684885 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:51.684844 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:51.685483 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:51.684914 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:51.690219 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:51.690193 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:52.161581 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:52.161553 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:00:54.444805 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:54.444769 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56687ff765-cdk8m"] Jun 11 15:00:57.185082 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:57.185036 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-g2qng" event={"ID":"0b2a36ef-be5f-4768-8925-891462a9549a","Type":"ContainerStarted","Data":"e789716bbef72f7a9819ce8a4755cbf4f4213dbf94f9644b8140a95c65c31099"} Jun 11 15:00:57.185558 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:57.185241 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-g2qng" Jun 11 15:00:57.197573 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:57.197547 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-g2qng" Jun 11 15:00:57.205218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:00:57.205165 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-g2qng" podStartSLOduration=1.6595179 podStartE2EDuration="23.205146176s" totalImagesPullingTime="21.545628276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:00:57.202536141 +0000 UTC m=+113.979167863" watchObservedRunningTime="2026-06-11 15:00:57.205146176 +0000 UTC m=+113.981777898" Jun 11 15:01:19.469629 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.469574 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56687ff765-cdk8m" podUID="7a8a9db5-dccd-4a3e-94b7-c823c61a4132" containerName="console" containerID="cri-o://c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c" gracePeriod=15 Jun 11 15:01:19.716205 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.716177 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:01:19.864340 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.864310 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-oauth-config\") pod \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " Jun 11 15:01:19.864504 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.864351 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-service-ca\") pod \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " Jun 11 15:01:19.864504 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.864384 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-serving-cert\") pod \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " Jun 11 15:01:19.864504 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.864439 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pktx9\" (UniqueName: \"kubernetes.io/projected/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-kube-api-access-pktx9\") pod \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " Jun 11 15:01:19.864504 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.864470 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-oauth-serving-cert\") pod \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " Jun 11 15:01:19.864708 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.864530 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-config\") pod \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\" (UID: \"7a8a9db5-dccd-4a3e-94b7-c823c61a4132\") " Jun 11 15:01:19.864929 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.864902 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-service-ca" (OuterVolumeSpecName: "service-ca") pod "7a8a9db5-dccd-4a3e-94b7-c823c61a4132" (UID: "7a8a9db5-dccd-4a3e-94b7-c823c61a4132"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:01:19.865063 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.865029 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7a8a9db5-dccd-4a3e-94b7-c823c61a4132" (UID: "7a8a9db5-dccd-4a3e-94b7-c823c61a4132"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:01:19.865063 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.865036 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-config" (OuterVolumeSpecName: "console-config") pod "7a8a9db5-dccd-4a3e-94b7-c823c61a4132" (UID: "7a8a9db5-dccd-4a3e-94b7-c823c61a4132"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:01:19.866681 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.866650 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7a8a9db5-dccd-4a3e-94b7-c823c61a4132" (UID: "7a8a9db5-dccd-4a3e-94b7-c823c61a4132"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:01:19.866681 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.866669 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7a8a9db5-dccd-4a3e-94b7-c823c61a4132" (UID: "7a8a9db5-dccd-4a3e-94b7-c823c61a4132"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:01:19.866837 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.866704 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-kube-api-access-pktx9" (OuterVolumeSpecName: "kube-api-access-pktx9") pod "7a8a9db5-dccd-4a3e-94b7-c823c61a4132" (UID: "7a8a9db5-dccd-4a3e-94b7-c823c61a4132"). InnerVolumeSpecName "kube-api-access-pktx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:01:19.966043 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.966007 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pktx9\" (UniqueName: \"kubernetes.io/projected/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-kube-api-access-pktx9\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:01:19.966043 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.966037 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-oauth-serving-cert\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:01:19.966043 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.966047 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-config\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:01:19.966265 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.966055 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-oauth-config\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:01:19.966265 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.966064 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-service-ca\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:01:19.966265 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:19.966072 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a8a9db5-dccd-4a3e-94b7-c823c61a4132-console-serving-cert\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:01:20.271096 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.271010 2562 generic.go:356] "Generic (PLEG): container finished" podID="7a8a9db5-dccd-4a3e-94b7-c823c61a4132" containerID="c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c" exitCode=0 Jun 11 15:01:20.271096 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.271053 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56687ff765-cdk8m" Jun 11 15:01:20.271285 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.271054 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56687ff765-cdk8m" event={"ID":"7a8a9db5-dccd-4a3e-94b7-c823c61a4132","Type":"ContainerDied","Data":"c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c"} Jun 11 15:01:20.271285 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.271179 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56687ff765-cdk8m" event={"ID":"7a8a9db5-dccd-4a3e-94b7-c823c61a4132","Type":"ContainerDied","Data":"c5d0927746c3b0bb3d5b7debf835bb3821fbefecec0e1c81801d9f6d15231dd2"} Jun 11 15:01:20.271285 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.271217 2562 scope.go:117] "RemoveContainer" containerID="c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c" Jun 11 15:01:20.279168 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.279149 2562 scope.go:117] "RemoveContainer" containerID="c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c" Jun 11 15:01:20.279401 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:01:20.279379 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c\": container with ID starting with c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c not found: ID does not exist" containerID="c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c" Jun 11 15:01:20.279474 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.279412 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c"} err="failed to get container status \"c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c\": rpc error: code = NotFound desc = could not find container \"c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c\": container with ID starting with c583d5db0a645431020ced9493944087746a612191e30de21fa8cd416e133b7c not found: ID does not exist" Jun 11 15:01:20.292407 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.292387 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56687ff765-cdk8m"] Jun 11 15:01:20.295637 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:20.295612 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56687ff765-cdk8m"] Jun 11 15:01:21.674028 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:21.673999 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a8a9db5-dccd-4a3e-94b7-c823c61a4132" path="/var/lib/kubelet/pods/7a8a9db5-dccd-4a3e-94b7-c823c61a4132/volumes" Jun 11 15:01:53.976982 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.976893 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6767c5dcdd-wfpbj"] Jun 11 15:01:53.977481 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.977269 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a8a9db5-dccd-4a3e-94b7-c823c61a4132" containerName="console" Jun 11 15:01:53.977481 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.977288 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a8a9db5-dccd-4a3e-94b7-c823c61a4132" containerName="console" Jun 11 15:01:53.977481 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.977378 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a8a9db5-dccd-4a3e-94b7-c823c61a4132" containerName="console" Jun 11 15:01:53.980026 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.980004 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:53.982937 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.982916 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 15:01:53.983039 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.982918 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 15:01:53.983039 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.982953 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 15:01:53.983039 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.982953 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 15:01:53.984421 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.984402 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 15:01:53.984516 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.984483 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f29lk\"" Jun 11 15:01:53.989902 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.989882 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 11 15:01:53.991828 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:53.991704 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6767c5dcdd-wfpbj"] Jun 11 15:01:54.013979 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.013958 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-oauth-config\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.014091 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.013987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-oauth-serving-cert\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.014091 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.014023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-serving-cert\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.014091 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.014068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-service-ca\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.014193 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.014101 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-trusted-ca-bundle\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.014193 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.014138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-console-config\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.014193 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.014172 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56v7r\" (UniqueName: \"kubernetes.io/projected/504a2896-8336-49d9-b784-b69701d56e8c-kube-api-access-56v7r\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.114995 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.114961 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-service-ca\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.114995 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.114996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-trusted-ca-bundle\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.115244 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-console-config\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.115244 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56v7r\" (UniqueName: \"kubernetes.io/projected/504a2896-8336-49d9-b784-b69701d56e8c-kube-api-access-56v7r\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.115244 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-oauth-config\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.115244 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-oauth-serving-cert\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.115244 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115149 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-serving-cert\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.115897 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115867 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-service-ca\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.116027 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115916 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-console-config\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.116027 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115941 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-oauth-serving-cert\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.116027 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.115975 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-trusted-ca-bundle\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.117580 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.117557 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-serving-cert\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.117580 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.117577 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-oauth-config\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.122795 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.122770 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56v7r\" (UniqueName: \"kubernetes.io/projected/504a2896-8336-49d9-b784-b69701d56e8c-kube-api-access-56v7r\") pod \"console-6767c5dcdd-wfpbj\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.289172 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.289075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:01:54.424126 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:54.424061 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6767c5dcdd-wfpbj"] Jun 11 15:01:54.426423 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:01:54.426396 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504a2896_8336_49d9_b784_b69701d56e8c.slice/crio-d8ba9f2133d4dafe424e0eb8d4fff50348cb7d6d7abf058284f133383306e751 WatchSource:0}: Error finding container d8ba9f2133d4dafe424e0eb8d4fff50348cb7d6d7abf058284f133383306e751: Status 404 returned error can't find the container with id d8ba9f2133d4dafe424e0eb8d4fff50348cb7d6d7abf058284f133383306e751 Jun 11 15:01:55.393991 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:55.393956 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6767c5dcdd-wfpbj" event={"ID":"504a2896-8336-49d9-b784-b69701d56e8c","Type":"ContainerStarted","Data":"a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544"} Jun 11 15:01:55.394458 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:55.394002 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6767c5dcdd-wfpbj" event={"ID":"504a2896-8336-49d9-b784-b69701d56e8c","Type":"ContainerStarted","Data":"d8ba9f2133d4dafe424e0eb8d4fff50348cb7d6d7abf058284f133383306e751"} Jun 11 15:01:55.410800 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:01:55.410749 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6767c5dcdd-wfpbj" podStartSLOduration=2.4107334639999998 podStartE2EDuration="2.410733464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:01:55.409880838 +0000 UTC m=+172.186512558" watchObservedRunningTime="2026-06-11 15:01:55.410733464 +0000 UTC m=+172.187365182" Jun 11 15:02:04.289523 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:02:04.289481 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:02:04.289990 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:02:04.289558 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:02:04.294202 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:02:04.294178 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:02:04.429635 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:02:04.429611 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:03:03.289701 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.289660 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84764fc5b4-w42td"] Jun 11 15:03:03.291843 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.291823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.303916 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.303890 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84764fc5b4-w42td"] Jun 11 15:03:03.330071 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.330042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-serving-cert\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.330213 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.330075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-service-ca\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.330213 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.330093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9xll\" (UniqueName: \"kubernetes.io/projected/6f156381-d740-434b-8afc-fe29012ae539-kube-api-access-z9xll\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.330213 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.330113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-console-config\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.330378 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.330206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-oauth-config\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.330378 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.330249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-trusted-ca-bundle\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.330378 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.330296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-oauth-serving-cert\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.431576 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.431542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-oauth-config\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.431576 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.431579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-trusted-ca-bundle\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.431808 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.431602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-oauth-serving-cert\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.431808 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.431630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-serving-cert\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.431808 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.431648 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-service-ca\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.431808 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.431666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9xll\" (UniqueName: \"kubernetes.io/projected/6f156381-d740-434b-8afc-fe29012ae539-kube-api-access-z9xll\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.431808 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.431686 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-console-config\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.432357 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.432333 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-oauth-serving-cert\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.432481 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.432420 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-service-ca\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.432481 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.432451 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-console-config\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.432607 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.432585 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-trusted-ca-bundle\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.434243 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.434212 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-oauth-config\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.434243 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.434229 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-serving-cert\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.440460 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.440442 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9xll\" (UniqueName: \"kubernetes.io/projected/6f156381-d740-434b-8afc-fe29012ae539-kube-api-access-z9xll\") pod \"console-84764fc5b4-w42td\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.601048 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.601007 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:03.727577 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:03.727411 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84764fc5b4-w42td"] Jun 11 15:03:03.729988 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:03:03.729961 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f156381_d740_434b_8afc_fe29012ae539.slice/crio-e036950b533de604d753577b70058c5c42790f5325a2a344024c6e4127b92b20 WatchSource:0}: Error finding container e036950b533de604d753577b70058c5c42790f5325a2a344024c6e4127b92b20: Status 404 returned error can't find the container with id e036950b533de604d753577b70058c5c42790f5325a2a344024c6e4127b92b20 Jun 11 15:03:04.633776 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:04.633737 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84764fc5b4-w42td" event={"ID":"6f156381-d740-434b-8afc-fe29012ae539","Type":"ContainerStarted","Data":"0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219"} Jun 11 15:03:04.634173 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:04.633784 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84764fc5b4-w42td" event={"ID":"6f156381-d740-434b-8afc-fe29012ae539","Type":"ContainerStarted","Data":"e036950b533de604d753577b70058c5c42790f5325a2a344024c6e4127b92b20"} Jun 11 15:03:04.653129 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:04.653078 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84764fc5b4-w42td" podStartSLOduration=1.6530631329999999 podStartE2EDuration="1.653063133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:03:04.651645031 +0000 UTC m=+241.428276752" watchObservedRunningTime="2026-06-11 15:03:04.653063133 +0000 UTC m=+241.429694853" Jun 11 15:03:13.601977 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:13.601897 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:13.602393 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:13.601980 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:13.606699 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:13.606680 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:13.668187 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:13.668159 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:03:13.714525 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:13.714494 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6767c5dcdd-wfpbj"] Jun 11 15:03:38.736315 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:38.736279 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6767c5dcdd-wfpbj" podUID="504a2896-8336-49d9-b784-b69701d56e8c" containerName="console" containerID="cri-o://a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544" gracePeriod=15 Jun 11 15:03:38.977269 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:38.977247 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:03:39.111792 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.111768 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-oauth-serving-cert\") pod \"504a2896-8336-49d9-b784-b69701d56e8c\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " Jun 11 15:03:39.111959 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.111826 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-console-config\") pod \"504a2896-8336-49d9-b784-b69701d56e8c\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " Jun 11 15:03:39.111959 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.111859 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-trusted-ca-bundle\") pod \"504a2896-8336-49d9-b784-b69701d56e8c\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " Jun 11 15:03:39.111959 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.111876 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-serving-cert\") pod \"504a2896-8336-49d9-b784-b69701d56e8c\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " Jun 11 15:03:39.111959 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.111893 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-oauth-config\") pod \"504a2896-8336-49d9-b784-b69701d56e8c\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " Jun 11 15:03:39.111959 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.111909 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56v7r\" (UniqueName: \"kubernetes.io/projected/504a2896-8336-49d9-b784-b69701d56e8c-kube-api-access-56v7r\") pod \"504a2896-8336-49d9-b784-b69701d56e8c\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " Jun 11 15:03:39.111959 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.111932 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-service-ca\") pod \"504a2896-8336-49d9-b784-b69701d56e8c\" (UID: \"504a2896-8336-49d9-b784-b69701d56e8c\") " Jun 11 15:03:39.112228 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.112172 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "504a2896-8336-49d9-b784-b69701d56e8c" (UID: "504a2896-8336-49d9-b784-b69701d56e8c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:03:39.112375 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.112350 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "504a2896-8336-49d9-b784-b69701d56e8c" (UID: "504a2896-8336-49d9-b784-b69701d56e8c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:03:39.112457 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.112433 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-service-ca" (OuterVolumeSpecName: "service-ca") pod "504a2896-8336-49d9-b784-b69701d56e8c" (UID: "504a2896-8336-49d9-b784-b69701d56e8c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:03:39.112502 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.112435 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-console-config" (OuterVolumeSpecName: "console-config") pod "504a2896-8336-49d9-b784-b69701d56e8c" (UID: "504a2896-8336-49d9-b784-b69701d56e8c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:03:39.114243 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.114216 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "504a2896-8336-49d9-b784-b69701d56e8c" (UID: "504a2896-8336-49d9-b784-b69701d56e8c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:03:39.114338 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.114245 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "504a2896-8336-49d9-b784-b69701d56e8c" (UID: "504a2896-8336-49d9-b784-b69701d56e8c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:03:39.114338 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.114257 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504a2896-8336-49d9-b784-b69701d56e8c-kube-api-access-56v7r" (OuterVolumeSpecName: "kube-api-access-56v7r") pod "504a2896-8336-49d9-b784-b69701d56e8c" (UID: "504a2896-8336-49d9-b784-b69701d56e8c"). InnerVolumeSpecName "kube-api-access-56v7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:03:39.212650 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.212616 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-trusted-ca-bundle\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:03:39.212650 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.212642 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-serving-cert\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:03:39.212650 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.212652 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/504a2896-8336-49d9-b784-b69701d56e8c-console-oauth-config\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:03:39.212650 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.212661 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56v7r\" (UniqueName: \"kubernetes.io/projected/504a2896-8336-49d9-b784-b69701d56e8c-kube-api-access-56v7r\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:03:39.212921 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.212670 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-service-ca\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:03:39.212921 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.212679 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-oauth-serving-cert\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:03:39.212921 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.212689 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/504a2896-8336-49d9-b784-b69701d56e8c-console-config\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:03:39.753852 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.753817 2562 generic.go:356] "Generic (PLEG): container finished" podID="504a2896-8336-49d9-b784-b69701d56e8c" containerID="a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544" exitCode=0 Jun 11 15:03:39.754260 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.753862 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6767c5dcdd-wfpbj" Jun 11 15:03:39.754260 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.753904 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6767c5dcdd-wfpbj" event={"ID":"504a2896-8336-49d9-b784-b69701d56e8c","Type":"ContainerDied","Data":"a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544"} Jun 11 15:03:39.754260 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.753958 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6767c5dcdd-wfpbj" event={"ID":"504a2896-8336-49d9-b784-b69701d56e8c","Type":"ContainerDied","Data":"d8ba9f2133d4dafe424e0eb8d4fff50348cb7d6d7abf058284f133383306e751"} Jun 11 15:03:39.754260 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.753983 2562 scope.go:117] "RemoveContainer" containerID="a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544" Jun 11 15:03:39.761765 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.761747 2562 scope.go:117] "RemoveContainer" containerID="a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544" Jun 11 15:03:39.762007 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:03:39.761988 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544\": container with ID starting with a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544 not found: ID does not exist" containerID="a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544" Jun 11 15:03:39.762053 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.762014 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544"} err="failed to get container status \"a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544\": rpc error: code = NotFound desc = could not find container \"a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544\": container with ID starting with a7a6b18f1ea13f7632c7a3e51305f7aa09aa58f95e9ef8eeb165626615342544 not found: ID does not exist" Jun 11 15:03:39.774394 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.774374 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6767c5dcdd-wfpbj"] Jun 11 15:03:39.779942 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:39.779922 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6767c5dcdd-wfpbj"] Jun 11 15:03:41.673632 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:03:41.673598 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504a2896-8336-49d9-b784-b69701d56e8c" path="/var/lib/kubelet/pods/504a2896-8336-49d9-b784-b69701d56e8c/volumes" Jun 11 15:04:03.617489 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:04:03.617455 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 15:04:03.618045 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:04:03.618028 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 15:04:03.629332 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:04:03.629310 2562 kubelet.go:1648] "Image garbage collection succeeded" Jun 11 15:05:25.189564 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.189531 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-57nc5"] Jun 11 15:05:25.190040 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.189809 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504a2896-8336-49d9-b784-b69701d56e8c" containerName="console" Jun 11 15:05:25.190040 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.189820 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="504a2896-8336-49d9-b784-b69701d56e8c" containerName="console" Jun 11 15:05:25.190040 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.189861 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="504a2896-8336-49d9-b784-b69701d56e8c" containerName="console" Jun 11 15:05:25.191646 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.191630 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-57nc5" Jun 11 15:05:25.194381 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.194357 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 11 15:05:25.194612 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.194596 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 11 15:05:25.195601 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.195586 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kl8tq\"" Jun 11 15:05:25.200355 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.200336 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg6p7\" (UniqueName: \"kubernetes.io/projected/dffab875-6947-4b9a-9c17-35ea03b167f3-kube-api-access-rg6p7\") pod \"cert-manager-59c77684cc-57nc5\" (UID: \"dffab875-6947-4b9a-9c17-35ea03b167f3\") " pod="cert-manager/cert-manager-59c77684cc-57nc5" Jun 11 15:05:25.200458 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.200385 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dffab875-6947-4b9a-9c17-35ea03b167f3-bound-sa-token\") pod \"cert-manager-59c77684cc-57nc5\" (UID: \"dffab875-6947-4b9a-9c17-35ea03b167f3\") " pod="cert-manager/cert-manager-59c77684cc-57nc5" Jun 11 15:05:25.261024 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.260989 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-57nc5"] Jun 11 15:05:25.301330 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.301294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dffab875-6947-4b9a-9c17-35ea03b167f3-bound-sa-token\") pod \"cert-manager-59c77684cc-57nc5\" (UID: \"dffab875-6947-4b9a-9c17-35ea03b167f3\") " pod="cert-manager/cert-manager-59c77684cc-57nc5" Jun 11 15:05:25.301503 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.301357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg6p7\" (UniqueName: \"kubernetes.io/projected/dffab875-6947-4b9a-9c17-35ea03b167f3-kube-api-access-rg6p7\") pod \"cert-manager-59c77684cc-57nc5\" (UID: \"dffab875-6947-4b9a-9c17-35ea03b167f3\") " pod="cert-manager/cert-manager-59c77684cc-57nc5" Jun 11 15:05:25.319159 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.319128 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg6p7\" (UniqueName: \"kubernetes.io/projected/dffab875-6947-4b9a-9c17-35ea03b167f3-kube-api-access-rg6p7\") pod \"cert-manager-59c77684cc-57nc5\" (UID: \"dffab875-6947-4b9a-9c17-35ea03b167f3\") " pod="cert-manager/cert-manager-59c77684cc-57nc5" Jun 11 15:05:25.320235 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.320218 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dffab875-6947-4b9a-9c17-35ea03b167f3-bound-sa-token\") pod \"cert-manager-59c77684cc-57nc5\" (UID: \"dffab875-6947-4b9a-9c17-35ea03b167f3\") " pod="cert-manager/cert-manager-59c77684cc-57nc5" Jun 11 15:05:25.500065 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.499972 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-57nc5" Jun 11 15:05:25.700436 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.700409 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-57nc5"] Jun 11 15:05:25.703949 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:05:25.703922 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddffab875_6947_4b9a_9c17_35ea03b167f3.slice/crio-e9ab66487509424e90e121aa0f8bc922eaf946d7728663077c2f34693480596a WatchSource:0}: Error finding container e9ab66487509424e90e121aa0f8bc922eaf946d7728663077c2f34693480596a: Status 404 returned error can't find the container with id e9ab66487509424e90e121aa0f8bc922eaf946d7728663077c2f34693480596a Jun 11 15:05:25.705632 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:25.705617 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 15:05:26.116873 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:26.116833 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-57nc5" event={"ID":"dffab875-6947-4b9a-9c17-35ea03b167f3","Type":"ContainerStarted","Data":"e9ab66487509424e90e121aa0f8bc922eaf946d7728663077c2f34693480596a"} Jun 11 15:05:29.134111 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:29.134077 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-57nc5" event={"ID":"dffab875-6947-4b9a-9c17-35ea03b167f3","Type":"ContainerStarted","Data":"7a818d060b0c7cdb75b276e2750bdda8beb90fd84b7f5eb0e3ce35860511a0f0"} Jun 11 15:05:46.668524 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.668468 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-57nc5" podStartSLOduration=18.684141411 podStartE2EDuration="21.66845201s" totalImagesPullingTime="2.984310599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:05:29.195118506 +0000 UTC m=+385.971750227" watchObservedRunningTime="2026-06-11 15:05:46.66845201 +0000 UTC m=+403.445083731" Jun 11 15:05:46.669547 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.669530 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh"] Jun 11 15:05:46.671951 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.671932 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.676938 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.676910 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 11 15:05:46.677049 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.676945 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 11 15:05:46.677049 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.676911 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 11 15:05:46.678091 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.677844 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 11 15:05:46.678386 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.678367 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w5czj\"" Jun 11 15:05:46.762659 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.762624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63aa2c07-839a-4e40-8dfe-06bb7782045e-tmp\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.762842 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.762668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k8m5\" (UniqueName: \"kubernetes.io/projected/63aa2c07-839a-4e40-8dfe-06bb7782045e-kube-api-access-2k8m5\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.762842 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.762709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/63aa2c07-839a-4e40-8dfe-06bb7782045e-manifests\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.762842 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.762815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63aa2c07-839a-4e40-8dfe-06bb7782045e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.763005 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.762877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63aa2c07-839a-4e40-8dfe-06bb7782045e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.863588 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.863559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k8m5\" (UniqueName: \"kubernetes.io/projected/63aa2c07-839a-4e40-8dfe-06bb7782045e-kube-api-access-2k8m5\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.863771 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.863599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/63aa2c07-839a-4e40-8dfe-06bb7782045e-manifests\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.863771 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.863635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63aa2c07-839a-4e40-8dfe-06bb7782045e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.863771 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.863663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63aa2c07-839a-4e40-8dfe-06bb7782045e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.863771 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.863680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63aa2c07-839a-4e40-8dfe-06bb7782045e-tmp\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.864198 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.864179 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/63aa2c07-839a-4e40-8dfe-06bb7782045e-manifests\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.866338 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.866313 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63aa2c07-839a-4e40-8dfe-06bb7782045e-tmp\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.866440 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.866428 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63aa2c07-839a-4e40-8dfe-06bb7782045e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.866440 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.866434 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63aa2c07-839a-4e40-8dfe-06bb7782045e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.901558 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.901526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k8m5\" (UniqueName: \"kubernetes.io/projected/63aa2c07-839a-4e40-8dfe-06bb7782045e-kube-api-access-2k8m5\") pod \"opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh\" (UID: \"63aa2c07-839a-4e40-8dfe-06bb7782045e\") " pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:46.942091 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.942056 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh"] Jun 11 15:05:46.981652 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:46.981616 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:05:47.137996 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:47.137868 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh"] Jun 11 15:05:47.144576 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:05:47.144543 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63aa2c07_839a_4e40_8dfe_06bb7782045e.slice/crio-25437ca1ff7c41ea2a96d8e200630e1d49ddd72f0ce82c7c066757a44759aec3 WatchSource:0}: Error finding container 25437ca1ff7c41ea2a96d8e200630e1d49ddd72f0ce82c7c066757a44759aec3: Status 404 returned error can't find the container with id 25437ca1ff7c41ea2a96d8e200630e1d49ddd72f0ce82c7c066757a44759aec3 Jun 11 15:05:47.199511 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:47.199478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" event={"ID":"63aa2c07-839a-4e40-8dfe-06bb7782045e","Type":"ContainerStarted","Data":"25437ca1ff7c41ea2a96d8e200630e1d49ddd72f0ce82c7c066757a44759aec3"} Jun 11 15:05:51.215977 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:51.215944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" event={"ID":"63aa2c07-839a-4e40-8dfe-06bb7782045e","Type":"ContainerStarted","Data":"6f0d8cb4ccfd02959f3826deb2cb91f74e9a2a160d2cd25e3290599f5df4bdac"} Jun 11 15:05:52.221422 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:52.221382 2562 generic.go:356] "Generic (PLEG): container finished" podID="63aa2c07-839a-4e40-8dfe-06bb7782045e" containerID="6f0d8cb4ccfd02959f3826deb2cb91f74e9a2a160d2cd25e3290599f5df4bdac" exitCode=0 Jun 11 15:05:52.221927 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:52.221439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" event={"ID":"63aa2c07-839a-4e40-8dfe-06bb7782045e","Type":"ContainerDied","Data":"6f0d8cb4ccfd02959f3826deb2cb91f74e9a2a160d2cd25e3290599f5df4bdac"} Jun 11 15:05:53.227217 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:53.227182 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" event={"ID":"63aa2c07-839a-4e40-8dfe-06bb7782045e","Type":"ContainerStarted","Data":"3c209b29bcdc9165c77a47dba748abc75ae75952e6409c6fd5fcaeb2f9954fd4"} Jun 11 15:05:53.227597 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:05:53.227305 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:06:04.233650 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:04.233620 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" Jun 11 15:06:04.280574 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:04.280522 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh" podStartSLOduration=13.442662287 podStartE2EDuration="18.280506266s" totalImagesPullingTime="3.848173099s" totalInitContainerRuntime="989.67088ms" isStatefulPod=false podCreationTimestamp="2026-06-11 15:05:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:05:53.269063386 +0000 UTC m=+410.045695106" watchObservedRunningTime="2026-06-11 15:06:04.280506266 +0000 UTC m=+421.057138031" Jun 11 15:06:06.817749 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.817701 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d"] Jun 11 15:06:06.830690 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.830668 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:06.839225 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.839206 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 11 15:06:06.839497 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.839478 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 11 15:06:06.839497 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.839494 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 11 15:06:06.839664 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.839494 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8788p\"" Jun 11 15:06:06.839664 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.839570 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 11 15:06:06.839815 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.839793 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 11 15:06:06.878943 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.878908 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d"] Jun 11 15:06:06.934457 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.934423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccbae6f2-de45-465d-83bc-bf4fb96a089b-metrics-cert\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:06.934644 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.934472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ccbae6f2-de45-465d-83bc-bf4fb96a089b-manager-config\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:06.934644 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.934543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccbae6f2-de45-465d-83bc-bf4fb96a089b-cert\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:06.934644 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:06.934568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp4zt\" (UniqueName: \"kubernetes.io/projected/ccbae6f2-de45-465d-83bc-bf4fb96a089b-kube-api-access-dp4zt\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.035608 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.035570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccbae6f2-de45-465d-83bc-bf4fb96a089b-metrics-cert\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.035824 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.035618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ccbae6f2-de45-465d-83bc-bf4fb96a089b-manager-config\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.035824 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.035662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccbae6f2-de45-465d-83bc-bf4fb96a089b-cert\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.035824 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.035689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp4zt\" (UniqueName: \"kubernetes.io/projected/ccbae6f2-de45-465d-83bc-bf4fb96a089b-kube-api-access-dp4zt\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.036378 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.036352 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ccbae6f2-de45-465d-83bc-bf4fb96a089b-manager-config\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.038009 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.037990 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccbae6f2-de45-465d-83bc-bf4fb96a089b-cert\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.038088 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.038038 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccbae6f2-de45-465d-83bc-bf4fb96a089b-metrics-cert\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.109471 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.109434 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp4zt\" (UniqueName: \"kubernetes.io/projected/ccbae6f2-de45-465d-83bc-bf4fb96a089b-kube-api-access-dp4zt\") pod \"lws-controller-manager-5f7555c9dd-mhf6d\" (UID: \"ccbae6f2-de45-465d-83bc-bf4fb96a089b\") " pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.139438 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.139407 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:07.594643 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:07.594608 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d"] Jun 11 15:06:07.616982 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:06:07.616956 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccbae6f2_de45_465d_83bc_bf4fb96a089b.slice/crio-14561cea646f4fe7c526c08732d5cec3a1eccbae8b071fa5d016772a9f4250af WatchSource:0}: Error finding container 14561cea646f4fe7c526c08732d5cec3a1eccbae8b071fa5d016772a9f4250af: Status 404 returned error can't find the container with id 14561cea646f4fe7c526c08732d5cec3a1eccbae8b071fa5d016772a9f4250af Jun 11 15:06:08.294588 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:08.294552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" event={"ID":"ccbae6f2-de45-465d-83bc-bf4fb96a089b","Type":"ContainerStarted","Data":"14561cea646f4fe7c526c08732d5cec3a1eccbae8b071fa5d016772a9f4250af"} Jun 11 15:06:11.310537 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:11.310441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" event={"ID":"ccbae6f2-de45-465d-83bc-bf4fb96a089b","Type":"ContainerStarted","Data":"31ed7f4af082fe5ed10edb5ddffb3944c0e5eec854ff0b9cb53f1b8599b61b99"} Jun 11 15:06:11.310537 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:11.310512 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:11.497279 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:11.497232 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" podStartSLOduration=2.225907276 podStartE2EDuration="5.497215229s" totalImagesPullingTime="3.271307953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:06:11.484811929 +0000 UTC m=+428.261443663" watchObservedRunningTime="2026-06-11 15:06:11.497215229 +0000 UTC m=+428.273846950" Jun 11 15:06:22.317087 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:22.317007 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5f7555c9dd-mhf6d" Jun 11 15:06:41.476962 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.476933 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-74d6667c75-8hgrn"] Jun 11 15:06:41.479245 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.479230 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" Jun 11 15:06:41.482686 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.482665 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 11 15:06:41.482823 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.482665 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kds79\"" Jun 11 15:06:41.496429 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.496401 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-74d6667c75-8hgrn"] Jun 11 15:06:41.503075 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.503050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f984a2c7-2a44-4d0b-ab25-65db6fb3a47d-redirect-config\") pod \"dashboard-redirect-74d6667c75-8hgrn\" (UID: \"f984a2c7-2a44-4d0b-ab25-65db6fb3a47d\") " pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" Jun 11 15:06:41.503209 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.503190 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxtwj\" (UniqueName: \"kubernetes.io/projected/f984a2c7-2a44-4d0b-ab25-65db6fb3a47d-kube-api-access-dxtwj\") pod \"dashboard-redirect-74d6667c75-8hgrn\" (UID: \"f984a2c7-2a44-4d0b-ab25-65db6fb3a47d\") " pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" Jun 11 15:06:41.604561 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.604520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxtwj\" (UniqueName: \"kubernetes.io/projected/f984a2c7-2a44-4d0b-ab25-65db6fb3a47d-kube-api-access-dxtwj\") pod \"dashboard-redirect-74d6667c75-8hgrn\" (UID: \"f984a2c7-2a44-4d0b-ab25-65db6fb3a47d\") " pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" Jun 11 15:06:41.604561 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.604566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f984a2c7-2a44-4d0b-ab25-65db6fb3a47d-redirect-config\") pod \"dashboard-redirect-74d6667c75-8hgrn\" (UID: \"f984a2c7-2a44-4d0b-ab25-65db6fb3a47d\") " pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" Jun 11 15:06:41.605229 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.605213 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f984a2c7-2a44-4d0b-ab25-65db6fb3a47d-redirect-config\") pod \"dashboard-redirect-74d6667c75-8hgrn\" (UID: \"f984a2c7-2a44-4d0b-ab25-65db6fb3a47d\") " pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" Jun 11 15:06:41.633841 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.633810 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxtwj\" (UniqueName: \"kubernetes.io/projected/f984a2c7-2a44-4d0b-ab25-65db6fb3a47d-kube-api-access-dxtwj\") pod \"dashboard-redirect-74d6667c75-8hgrn\" (UID: \"f984a2c7-2a44-4d0b-ab25-65db6fb3a47d\") " pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" Jun 11 15:06:41.789534 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.789446 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" Jun 11 15:06:41.925215 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:41.925190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-74d6667c75-8hgrn"] Jun 11 15:06:41.926900 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:06:41.926864 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf984a2c7_2a44_4d0b_ab25_65db6fb3a47d.slice/crio-34f7e51627c1c1649347fa44059e48033ea97235b1843d53840d92ac9ac19a75 WatchSource:0}: Error finding container 34f7e51627c1c1649347fa44059e48033ea97235b1843d53840d92ac9ac19a75: Status 404 returned error can't find the container with id 34f7e51627c1c1649347fa44059e48033ea97235b1843d53840d92ac9ac19a75 Jun 11 15:06:42.431044 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:42.431010 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" event={"ID":"f984a2c7-2a44-4d0b-ab25-65db6fb3a47d","Type":"ContainerStarted","Data":"34f7e51627c1c1649347fa44059e48033ea97235b1843d53840d92ac9ac19a75"} Jun 11 15:06:47.458920 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:47.458882 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" event={"ID":"f984a2c7-2a44-4d0b-ab25-65db6fb3a47d","Type":"ContainerStarted","Data":"3c40acfc0e75baf54153dc4712c15c2066997330b8de0a12470048aa207b2515"} Jun 11 15:06:47.485521 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:06:47.485471 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-74d6667c75-8hgrn" podStartSLOduration=1.902680801 podStartE2EDuration="6.485455686s" totalImagesPullingTime="4.582774885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:06:47.484266654 +0000 UTC m=+464.260898376" watchObservedRunningTime="2026-06-11 15:06:47.485455686 +0000 UTC m=+464.262087407" Jun 11 15:07:04.378390 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.378359 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz"] Jun 11 15:07:04.381372 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.381353 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.384022 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.384003 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 11 15:07:04.384165 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.384140 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zrzfq\"" Jun 11 15:07:04.393484 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.393459 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz"] Jun 11 15:07:04.498380 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.498380 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.498643 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.498643 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498557 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.498643 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.498643 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tht\" (UniqueName: \"kubernetes.io/projected/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-kube-api-access-s8tht\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.498855 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.498855 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.498855 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.498706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.599976 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.599940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tht\" (UniqueName: \"kubernetes.io/projected/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-kube-api-access-s8tht\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600369 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600369 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600182 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600369 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600369 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600568 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600396 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600568 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600704 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600685 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600793 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600770 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.600832 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.600792 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.602667 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.602635 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.602858 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.602831 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.608181 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.608161 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tht\" (UniqueName: \"kubernetes.io/projected/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-kube-api-access-s8tht\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.608631 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.608612 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44715b9a-f62e-4b39-b48d-4dd4c0b761a1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795sl7jz\" (UID: \"44715b9a-f62e-4b39-b48d-4dd4c0b761a1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.691012 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.690928 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:04.823456 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.823427 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5654b4ffc4-42z52"] Jun 11 15:07:04.827169 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.827144 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz"] Jun 11 15:07:04.827301 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.827237 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:04.830271 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:07:04.830251 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44715b9a_f62e_4b39_b48d_4dd4c0b761a1.slice/crio-2347c1156f38d017e103cb3d9f34bd897d78d2af12f6a4c011b41e6e742bc04d WatchSource:0}: Error finding container 2347c1156f38d017e103cb3d9f34bd897d78d2af12f6a4c011b41e6e742bc04d: Status 404 returned error can't find the container with id 2347c1156f38d017e103cb3d9f34bd897d78d2af12f6a4c011b41e6e742bc04d Jun 11 15:07:04.839577 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.839557 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5654b4ffc4-42z52"] Jun 11 15:07:04.902079 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.902053 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-service-ca\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:04.902233 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.902090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmmdt\" (UniqueName: \"kubernetes.io/projected/65bb7535-4585-49ad-bbeb-ff1837efc8b7-kube-api-access-fmmdt\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:04.902233 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.902112 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-serving-cert\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:04.902233 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.902154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-config\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:04.902233 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.902180 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-trusted-ca-bundle\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:04.902233 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.902209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-oauth-serving-cert\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:04.902395 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:04.902242 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-oauth-config\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.002899 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.002814 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-oauth-serving-cert\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.002899 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.002866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-oauth-config\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.002899 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.002899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-service-ca\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.003150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.002933 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmmdt\" (UniqueName: \"kubernetes.io/projected/65bb7535-4585-49ad-bbeb-ff1837efc8b7-kube-api-access-fmmdt\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.003150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.002956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-serving-cert\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.003150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.002980 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-config\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.003150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.003005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-trusted-ca-bundle\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.003673 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.003647 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-oauth-serving-cert\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.003673 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.003667 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-service-ca\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.003854 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.003770 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-trusted-ca-bundle\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.003854 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.003816 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-config\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.005316 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.005295 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-serving-cert\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.005408 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.005380 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65bb7535-4585-49ad-bbeb-ff1837efc8b7-console-oauth-config\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.018056 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.018033 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmmdt\" (UniqueName: \"kubernetes.io/projected/65bb7535-4585-49ad-bbeb-ff1837efc8b7-kube-api-access-fmmdt\") pod \"console-5654b4ffc4-42z52\" (UID: \"65bb7535-4585-49ad-bbeb-ff1837efc8b7\") " pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.145330 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.145299 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:05.272741 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.272657 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5654b4ffc4-42z52"] Jun 11 15:07:05.275577 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:07:05.275547 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65bb7535_4585_49ad_bbeb_ff1837efc8b7.slice/crio-931b61659c974342d19a6dc007ffcd0450d5eae3853e02d6448c8d8b3e9f1fdf WatchSource:0}: Error finding container 931b61659c974342d19a6dc007ffcd0450d5eae3853e02d6448c8d8b3e9f1fdf: Status 404 returned error can't find the container with id 931b61659c974342d19a6dc007ffcd0450d5eae3853e02d6448c8d8b3e9f1fdf Jun 11 15:07:05.532996 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.532901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" event={"ID":"44715b9a-f62e-4b39-b48d-4dd4c0b761a1","Type":"ContainerStarted","Data":"2347c1156f38d017e103cb3d9f34bd897d78d2af12f6a4c011b41e6e742bc04d"} Jun 11 15:07:05.534834 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.534761 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5654b4ffc4-42z52" event={"ID":"65bb7535-4585-49ad-bbeb-ff1837efc8b7","Type":"ContainerStarted","Data":"1aceb9ad1f2611722d01d7165f98bebdc442eaed071f5e6baaea5d6f1db2778d"} Jun 11 15:07:05.534834 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.534806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5654b4ffc4-42z52" event={"ID":"65bb7535-4585-49ad-bbeb-ff1837efc8b7","Type":"ContainerStarted","Data":"931b61659c974342d19a6dc007ffcd0450d5eae3853e02d6448c8d8b3e9f1fdf"} Jun 11 15:07:05.555193 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:05.555137 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5654b4ffc4-42z52" podStartSLOduration=1.555116669 podStartE2EDuration="1.555116669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:07:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:07:05.553044193 +0000 UTC m=+482.329675916" watchObservedRunningTime="2026-06-11 15:07:05.555116669 +0000 UTC m=+482.331748387" Jun 11 15:07:07.358561 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:07.358526 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:07:07.358823 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:07.358602 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:07:07.358823 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:07.358629 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:07:07.548923 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:07.548892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" event={"ID":"44715b9a-f62e-4b39-b48d-4dd4c0b761a1","Type":"ContainerStarted","Data":"2260316ff813627b0e32136195033b7c89fed3a84360d3741eeaf6a6c42deb47"} Jun 11 15:07:07.571350 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:07.571296 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" podStartSLOduration=1.04528096 podStartE2EDuration="3.571281363s" totalImagesPullingTime="2.526000403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:07:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:07:07.569346197 +0000 UTC m=+484.345977918" watchObservedRunningTime="2026-06-11 15:07:07.571281363 +0000 UTC m=+484.347913083" Jun 11 15:07:07.691920 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:07.691842 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:08.696046 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:08.696018 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:09.558751 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:09.558696 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:09.559635 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:09.559617 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795sl7jz" Jun 11 15:07:10.329652 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.329620 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m2qjb"] Jun 11 15:07:10.332109 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.332091 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" Jun 11 15:07:10.334757 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.334726 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lt5cx\"" Jun 11 15:07:10.334757 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.334751 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 15:07:10.336072 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.336049 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 15:07:10.345149 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.345125 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m2qjb"] Jun 11 15:07:10.453828 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.453795 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q4ng\" (UniqueName: \"kubernetes.io/projected/827effda-7720-4481-9ed6-47e9542ab512-kube-api-access-2q4ng\") pod \"kuadrant-operator-catalog-m2qjb\" (UID: \"827effda-7720-4481-9ed6-47e9542ab512\") " pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" Jun 11 15:07:10.554780 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.554745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q4ng\" (UniqueName: \"kubernetes.io/projected/827effda-7720-4481-9ed6-47e9542ab512-kube-api-access-2q4ng\") pod \"kuadrant-operator-catalog-m2qjb\" (UID: \"827effda-7720-4481-9ed6-47e9542ab512\") " pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" Jun 11 15:07:10.564277 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.564247 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q4ng\" (UniqueName: \"kubernetes.io/projected/827effda-7720-4481-9ed6-47e9542ab512-kube-api-access-2q4ng\") pod \"kuadrant-operator-catalog-m2qjb\" (UID: \"827effda-7720-4481-9ed6-47e9542ab512\") " pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" Jun 11 15:07:10.643069 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.643042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" Jun 11 15:07:10.694706 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.694676 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m2qjb"] Jun 11 15:07:10.792577 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.792552 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m2qjb"] Jun 11 15:07:10.794340 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:07:10.794313 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827effda_7720_4481_9ed6_47e9542ab512.slice/crio-37645e98a9fd399a09f32c0a80fa16c6f5f9ff8aa4fda629bcb6bb27eeaa1b05 WatchSource:0}: Error finding container 37645e98a9fd399a09f32c0a80fa16c6f5f9ff8aa4fda629bcb6bb27eeaa1b05: Status 404 returned error can't find the container with id 37645e98a9fd399a09f32c0a80fa16c6f5f9ff8aa4fda629bcb6bb27eeaa1b05 Jun 11 15:07:10.912754 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.912657 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7t6t"] Jun 11 15:07:10.919661 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.919628 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:10.925524 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:10.925495 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7t6t"] Jun 11 15:07:11.058867 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:11.058830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4pgv\" (UniqueName: \"kubernetes.io/projected/a0a2f432-9536-4e09-9aea-6fdc24fdc356-kube-api-access-j4pgv\") pod \"kuadrant-operator-catalog-h7t6t\" (UID: \"a0a2f432-9536-4e09-9aea-6fdc24fdc356\") " pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:11.159982 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:11.159949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4pgv\" (UniqueName: \"kubernetes.io/projected/a0a2f432-9536-4e09-9aea-6fdc24fdc356-kube-api-access-j4pgv\") pod \"kuadrant-operator-catalog-h7t6t\" (UID: \"a0a2f432-9536-4e09-9aea-6fdc24fdc356\") " pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:11.169099 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:11.169035 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4pgv\" (UniqueName: \"kubernetes.io/projected/a0a2f432-9536-4e09-9aea-6fdc24fdc356-kube-api-access-j4pgv\") pod \"kuadrant-operator-catalog-h7t6t\" (UID: \"a0a2f432-9536-4e09-9aea-6fdc24fdc356\") " pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:11.229426 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:11.229395 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:11.369638 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:11.369605 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h7t6t"] Jun 11 15:07:11.421061 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:07:11.420990 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a2f432_9536_4e09_9aea_6fdc24fdc356.slice/crio-9a3ea814824080b56ff8a3e0364dcbeb0e6bea94cb99f55239ea44938b37fa04 WatchSource:0}: Error finding container 9a3ea814824080b56ff8a3e0364dcbeb0e6bea94cb99f55239ea44938b37fa04: Status 404 returned error can't find the container with id 9a3ea814824080b56ff8a3e0364dcbeb0e6bea94cb99f55239ea44938b37fa04 Jun 11 15:07:11.568266 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:11.568228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" event={"ID":"827effda-7720-4481-9ed6-47e9542ab512","Type":"ContainerStarted","Data":"37645e98a9fd399a09f32c0a80fa16c6f5f9ff8aa4fda629bcb6bb27eeaa1b05"} Jun 11 15:07:11.569598 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:11.569567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" event={"ID":"a0a2f432-9536-4e09-9aea-6fdc24fdc356","Type":"ContainerStarted","Data":"9a3ea814824080b56ff8a3e0364dcbeb0e6bea94cb99f55239ea44938b37fa04"} Jun 11 15:07:13.580521 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:13.580429 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" event={"ID":"827effda-7720-4481-9ed6-47e9542ab512","Type":"ContainerStarted","Data":"ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6"} Jun 11 15:07:13.580521 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:13.580495 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" podUID="827effda-7720-4481-9ed6-47e9542ab512" containerName="registry-server" containerID="cri-o://ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6" gracePeriod=2 Jun 11 15:07:13.582025 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:13.582005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" event={"ID":"a0a2f432-9536-4e09-9aea-6fdc24fdc356","Type":"ContainerStarted","Data":"e3dd2aaa440f6cda3a2fdd0b6e6e39b52e734d397de97444809e5df301d4a925"} Jun 11 15:07:13.602292 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:13.602248 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" podStartSLOduration=1.244045646 podStartE2EDuration="3.60223405s" totalImagesPullingTime="2.358188404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:07:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:07:13.599956498 +0000 UTC m=+490.376588209" watchObservedRunningTime="2026-06-11 15:07:13.60223405 +0000 UTC m=+490.378865774" Jun 11 15:07:13.618337 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:13.618288 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" podStartSLOduration=1.891041173 podStartE2EDuration="3.618274423s" totalImagesPullingTime="1.72723325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:07:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:07:13.616846392 +0000 UTC m=+490.393478113" watchObservedRunningTime="2026-06-11 15:07:13.618274423 +0000 UTC m=+490.394906144" Jun 11 15:07:13.820664 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:13.820643 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" Jun 11 15:07:13.985425 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:13.985388 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q4ng\" (UniqueName: \"kubernetes.io/projected/827effda-7720-4481-9ed6-47e9542ab512-kube-api-access-2q4ng\") pod \"827effda-7720-4481-9ed6-47e9542ab512\" (UID: \"827effda-7720-4481-9ed6-47e9542ab512\") " Jun 11 15:07:13.987503 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:13.987478 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/827effda-7720-4481-9ed6-47e9542ab512-kube-api-access-2q4ng" (OuterVolumeSpecName: "kube-api-access-2q4ng") pod "827effda-7720-4481-9ed6-47e9542ab512" (UID: "827effda-7720-4481-9ed6-47e9542ab512"). InnerVolumeSpecName "kube-api-access-2q4ng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:07:14.086958 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.086927 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q4ng\" (UniqueName: \"kubernetes.io/projected/827effda-7720-4481-9ed6-47e9542ab512-kube-api-access-2q4ng\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:07:14.592328 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.592297 2562 generic.go:356] "Generic (PLEG): container finished" podID="827effda-7720-4481-9ed6-47e9542ab512" containerID="ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6" exitCode=0 Jun 11 15:07:14.592732 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.592349 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" Jun 11 15:07:14.592732 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.592386 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" event={"ID":"827effda-7720-4481-9ed6-47e9542ab512","Type":"ContainerDied","Data":"ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6"} Jun 11 15:07:14.592732 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.592461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m2qjb" event={"ID":"827effda-7720-4481-9ed6-47e9542ab512","Type":"ContainerDied","Data":"37645e98a9fd399a09f32c0a80fa16c6f5f9ff8aa4fda629bcb6bb27eeaa1b05"} Jun 11 15:07:14.592732 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.592485 2562 scope.go:117] "RemoveContainer" containerID="ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6" Jun 11 15:07:14.601092 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.601071 2562 scope.go:117] "RemoveContainer" containerID="ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6" Jun 11 15:07:14.601317 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:07:14.601298 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6\": container with ID starting with ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6 not found: ID does not exist" containerID="ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6" Jun 11 15:07:14.601377 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.601325 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6"} err="failed to get container status \"ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6\": rpc error: code = NotFound desc = could not find container \"ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6\": container with ID starting with ccde017b1b69cdf4471a6d40c2f120f175d2ed909e46031b4d1115fca7a0cde6 not found: ID does not exist" Jun 11 15:07:14.614475 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.614454 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m2qjb"] Jun 11 15:07:14.618116 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:14.618096 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m2qjb"] Jun 11 15:07:15.145429 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:15.145396 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:15.145624 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:15.145457 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:15.149831 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:15.149810 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:15.601071 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:15.601046 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5654b4ffc4-42z52" Jun 11 15:07:15.650575 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:15.650543 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84764fc5b4-w42td"] Jun 11 15:07:15.677321 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:15.677286 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="827effda-7720-4481-9ed6-47e9542ab512" path="/var/lib/kubelet/pods/827effda-7720-4481-9ed6-47e9542ab512/volumes" Jun 11 15:07:21.230142 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:21.230107 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:21.230553 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:21.230169 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:21.253240 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:21.253217 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:21.644100 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:21.644074 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-h7t6t" Jun 11 15:07:39.970146 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:39.970091 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn"] Jun 11 15:07:39.970617 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:39.970594 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827effda-7720-4481-9ed6-47e9542ab512" containerName="registry-server" Jun 11 15:07:39.970617 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:39.970613 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="827effda-7720-4481-9ed6-47e9542ab512" containerName="registry-server" Jun 11 15:07:39.970755 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:39.970733 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="827effda-7720-4481-9ed6-47e9542ab512" containerName="registry-server" Jun 11 15:07:39.986637 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:39.986609 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" Jun 11 15:07:39.990595 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:39.990571 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rqxkx\"" Jun 11 15:07:39.992007 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:39.991987 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 11 15:07:39.994130 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:39.994110 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn"] Jun 11 15:07:40.102175 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:40.102130 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlfkc\" (UniqueName: \"kubernetes.io/projected/6d35b04a-0e6d-45d0-82c0-189b14a9cba3-kube-api-access-xlfkc\") pod \"dns-operator-controller-manager-864dd78c4c-n59rn\" (UID: \"6d35b04a-0e6d-45d0-82c0-189b14a9cba3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" Jun 11 15:07:40.203084 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:40.203036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlfkc\" (UniqueName: \"kubernetes.io/projected/6d35b04a-0e6d-45d0-82c0-189b14a9cba3-kube-api-access-xlfkc\") pod \"dns-operator-controller-manager-864dd78c4c-n59rn\" (UID: \"6d35b04a-0e6d-45d0-82c0-189b14a9cba3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" Jun 11 15:07:40.213398 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:40.213368 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlfkc\" (UniqueName: \"kubernetes.io/projected/6d35b04a-0e6d-45d0-82c0-189b14a9cba3-kube-api-access-xlfkc\") pod \"dns-operator-controller-manager-864dd78c4c-n59rn\" (UID: \"6d35b04a-0e6d-45d0-82c0-189b14a9cba3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" Jun 11 15:07:40.298113 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:40.298018 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" Jun 11 15:07:40.427968 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:40.427942 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn"] Jun 11 15:07:40.430567 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:07:40.430541 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d35b04a_0e6d_45d0_82c0_189b14a9cba3.slice/crio-ee192877dd517f7423a09f98a3d6d38f8586d1031a162492a28187d07ce2b3aa WatchSource:0}: Error finding container ee192877dd517f7423a09f98a3d6d38f8586d1031a162492a28187d07ce2b3aa: Status 404 returned error can't find the container with id ee192877dd517f7423a09f98a3d6d38f8586d1031a162492a28187d07ce2b3aa Jun 11 15:07:40.672294 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:40.672261 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-84764fc5b4-w42td" podUID="6f156381-d740-434b-8afc-fe29012ae539" containerName="console" containerID="cri-o://0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219" gracePeriod=15 Jun 11 15:07:40.702630 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:40.702601 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" event={"ID":"6d35b04a-0e6d-45d0-82c0-189b14a9cba3","Type":"ContainerStarted","Data":"ee192877dd517f7423a09f98a3d6d38f8586d1031a162492a28187d07ce2b3aa"} Jun 11 15:07:40.903840 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:40.903814 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:07:41.010687 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.010621 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-trusted-ca-bundle\") pod \"6f156381-d740-434b-8afc-fe29012ae539\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " Jun 11 15:07:41.010687 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.010671 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-console-config\") pod \"6f156381-d740-434b-8afc-fe29012ae539\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " Jun 11 15:07:41.011198 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.010703 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9xll\" (UniqueName: \"kubernetes.io/projected/6f156381-d740-434b-8afc-fe29012ae539-kube-api-access-z9xll\") pod \"6f156381-d740-434b-8afc-fe29012ae539\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " Jun 11 15:07:41.011198 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.010855 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-oauth-serving-cert\") pod \"6f156381-d740-434b-8afc-fe29012ae539\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " Jun 11 15:07:41.011198 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.010908 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-service-ca\") pod \"6f156381-d740-434b-8afc-fe29012ae539\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " Jun 11 15:07:41.011198 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.010931 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-oauth-config\") pod \"6f156381-d740-434b-8afc-fe29012ae539\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " Jun 11 15:07:41.011198 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.010959 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-serving-cert\") pod \"6f156381-d740-434b-8afc-fe29012ae539\" (UID: \"6f156381-d740-434b-8afc-fe29012ae539\") " Jun 11 15:07:41.011198 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.011144 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6f156381-d740-434b-8afc-fe29012ae539" (UID: "6f156381-d740-434b-8afc-fe29012ae539"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:07:41.011198 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.011152 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-console-config" (OuterVolumeSpecName: "console-config") pod "6f156381-d740-434b-8afc-fe29012ae539" (UID: "6f156381-d740-434b-8afc-fe29012ae539"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:07:41.011546 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.011285 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6f156381-d740-434b-8afc-fe29012ae539" (UID: "6f156381-d740-434b-8afc-fe29012ae539"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:07:41.011546 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.011355 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-service-ca" (OuterVolumeSpecName: "service-ca") pod "6f156381-d740-434b-8afc-fe29012ae539" (UID: "6f156381-d740-434b-8afc-fe29012ae539"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 15:07:41.011546 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.011409 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-trusted-ca-bundle\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:07:41.011546 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.011429 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-console-config\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:07:41.011546 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.011442 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-oauth-serving-cert\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:07:41.013016 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.012996 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6f156381-d740-434b-8afc-fe29012ae539" (UID: "6f156381-d740-434b-8afc-fe29012ae539"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:07:41.013152 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.013130 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f156381-d740-434b-8afc-fe29012ae539-kube-api-access-z9xll" (OuterVolumeSpecName: "kube-api-access-z9xll") pod "6f156381-d740-434b-8afc-fe29012ae539" (UID: "6f156381-d740-434b-8afc-fe29012ae539"). InnerVolumeSpecName "kube-api-access-z9xll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:07:41.013219 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.013197 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6f156381-d740-434b-8afc-fe29012ae539" (UID: "6f156381-d740-434b-8afc-fe29012ae539"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:07:41.112427 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.112393 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9xll\" (UniqueName: \"kubernetes.io/projected/6f156381-d740-434b-8afc-fe29012ae539-kube-api-access-z9xll\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:07:41.112427 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.112426 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f156381-d740-434b-8afc-fe29012ae539-service-ca\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:07:41.112427 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.112435 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-oauth-config\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:07:41.112736 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.112444 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f156381-d740-434b-8afc-fe29012ae539-console-serving-cert\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:07:41.708173 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.708139 2562 generic.go:356] "Generic (PLEG): container finished" podID="6f156381-d740-434b-8afc-fe29012ae539" containerID="0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219" exitCode=0 Jun 11 15:07:41.708328 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.708183 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84764fc5b4-w42td" Jun 11 15:07:41.708328 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.708217 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84764fc5b4-w42td" event={"ID":"6f156381-d740-434b-8afc-fe29012ae539","Type":"ContainerDied","Data":"0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219"} Jun 11 15:07:41.708328 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.708278 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84764fc5b4-w42td" event={"ID":"6f156381-d740-434b-8afc-fe29012ae539","Type":"ContainerDied","Data":"e036950b533de604d753577b70058c5c42790f5325a2a344024c6e4127b92b20"} Jun 11 15:07:41.708328 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.708305 2562 scope.go:117] "RemoveContainer" containerID="0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219" Jun 11 15:07:41.716454 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.716437 2562 scope.go:117] "RemoveContainer" containerID="0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219" Jun 11 15:07:41.716726 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:07:41.716698 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219\": container with ID starting with 0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219 not found: ID does not exist" containerID="0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219" Jun 11 15:07:41.716838 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.716819 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219"} err="failed to get container status \"0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219\": rpc error: code = NotFound desc = could not find container \"0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219\": container with ID starting with 0ddbbe413b9a570aaa705d6140aa65de5643df9c05311a8714d5b3ea18ccf219 not found: ID does not exist" Jun 11 15:07:41.735352 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.735323 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84764fc5b4-w42td"] Jun 11 15:07:41.742778 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:41.742757 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84764fc5b4-w42td"] Jun 11 15:07:43.674994 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:43.674962 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f156381-d740-434b-8afc-fe29012ae539" path="/var/lib/kubelet/pods/6f156381-d740-434b-8afc-fe29012ae539/volumes" Jun 11 15:07:43.725686 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:43.725651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" event={"ID":"6d35b04a-0e6d-45d0-82c0-189b14a9cba3","Type":"ContainerStarted","Data":"3cca27dd02d1d99b6b9b73295a73911688939e0e45d9d60e2e8482397a93093f"} Jun 11 15:07:43.725867 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:43.725781 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" Jun 11 15:07:43.744857 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:43.744812 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" podStartSLOduration=2.151289294 podStartE2EDuration="4.74479896s" totalImagesPullingTime="2.593509666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:07:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:07:43.744484657 +0000 UTC m=+520.521116378" watchObservedRunningTime="2026-06-11 15:07:43.74479896 +0000 UTC m=+520.521430680" Jun 11 15:07:44.436633 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.436601 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf"] Jun 11 15:07:44.436930 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.436914 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f156381-d740-434b-8afc-fe29012ae539" containerName="console" Jun 11 15:07:44.436930 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.436925 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f156381-d740-434b-8afc-fe29012ae539" containerName="console" Jun 11 15:07:44.437039 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.436994 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f156381-d740-434b-8afc-fe29012ae539" containerName="console" Jun 11 15:07:44.439015 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.438999 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:07:44.441906 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.441887 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hgl4h\"" Jun 11 15:07:44.457502 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.457475 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf"] Jun 11 15:07:44.542225 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.542194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4kvd\" (UniqueName: \"kubernetes.io/projected/3c38e1ac-8fb7-4143-b154-521c4905159e-kube-api-access-m4kvd\") pod \"limitador-operator-controller-manager-859d7b78bd-q25hf\" (UID: \"3c38e1ac-8fb7-4143-b154-521c4905159e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:07:44.642876 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.642835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4kvd\" (UniqueName: \"kubernetes.io/projected/3c38e1ac-8fb7-4143-b154-521c4905159e-kube-api-access-m4kvd\") pod \"limitador-operator-controller-manager-859d7b78bd-q25hf\" (UID: \"3c38e1ac-8fb7-4143-b154-521c4905159e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:07:44.651891 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.651859 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4kvd\" (UniqueName: \"kubernetes.io/projected/3c38e1ac-8fb7-4143-b154-521c4905159e-kube-api-access-m4kvd\") pod \"limitador-operator-controller-manager-859d7b78bd-q25hf\" (UID: \"3c38e1ac-8fb7-4143-b154-521c4905159e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:07:44.748790 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.748685 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:07:44.878618 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:44.878594 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf"] Jun 11 15:07:44.881161 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:07:44.881130 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c38e1ac_8fb7_4143_b154_521c4905159e.slice/crio-ec797a1ea1a249509ec5000536051175c224ba9423e8b8446ca5e650296302c4 WatchSource:0}: Error finding container ec797a1ea1a249509ec5000536051175c224ba9423e8b8446ca5e650296302c4: Status 404 returned error can't find the container with id ec797a1ea1a249509ec5000536051175c224ba9423e8b8446ca5e650296302c4 Jun 11 15:07:45.736673 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:45.736642 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" event={"ID":"3c38e1ac-8fb7-4143-b154-521c4905159e","Type":"ContainerStarted","Data":"ec797a1ea1a249509ec5000536051175c224ba9423e8b8446ca5e650296302c4"} Jun 11 15:07:47.747686 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:47.747650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" event={"ID":"3c38e1ac-8fb7-4143-b154-521c4905159e","Type":"ContainerStarted","Data":"7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408"} Jun 11 15:07:47.748100 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:47.747766 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:07:47.765904 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:47.765860 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" podStartSLOduration=1.346800615 podStartE2EDuration="3.76584134s" totalImagesPullingTime="2.419040725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:07:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:07:47.765433184 +0000 UTC m=+524.542064906" watchObservedRunningTime="2026-06-11 15:07:47.76584134 +0000 UTC m=+524.542473064" Jun 11 15:07:54.732546 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:54.732517 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-n59rn" Jun 11 15:07:58.754217 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:07:58.754188 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:01.557169 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.557133 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7"] Jun 11 15:08:01.561096 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.561078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.563706 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.563689 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 11 15:08:01.564384 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.564360 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 11 15:08:01.564596 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.564577 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hdp2v\"" Jun 11 15:08:01.570399 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.570376 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7"] Jun 11 15:08:01.686577 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.686549 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.686767 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.686587 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhc8j\" (UniqueName: \"kubernetes.io/projected/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-kube-api-access-rhc8j\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.686767 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.686665 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.788021 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.787985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.788199 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.788094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.788199 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.788151 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhc8j\" (UniqueName: \"kubernetes.io/projected/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-kube-api-access-rhc8j\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.788806 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.788788 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.790382 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.790362 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.797361 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.797339 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhc8j\" (UniqueName: \"kubernetes.io/projected/e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c-kube-api-access-rhc8j\") pod \"kuadrant-console-plugin-66b5774985-2zlx7\" (UID: \"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:01.870675 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:01.870647 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" Jun 11 15:08:02.006784 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:02.006757 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7"] Jun 11 15:08:02.007536 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:08:02.007515 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8ee22a4_188d_43fe_8a49_e5c9bbfa6f4c.slice/crio-f21af5a77d8dd0496c0b550e17873d03748b8bd743c4bc146d8f6621d265e6ab WatchSource:0}: Error finding container f21af5a77d8dd0496c0b550e17873d03748b8bd743c4bc146d8f6621d265e6ab: Status 404 returned error can't find the container with id f21af5a77d8dd0496c0b550e17873d03748b8bd743c4bc146d8f6621d265e6ab Jun 11 15:08:02.812244 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:02.812204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" event={"ID":"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c","Type":"ContainerStarted","Data":"f21af5a77d8dd0496c0b550e17873d03748b8bd743c4bc146d8f6621d265e6ab"} Jun 11 15:08:12.215590 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.215556 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr"] Jun 11 15:08:12.219311 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.219289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:12.223683 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.223660 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2cm2t\"" Jun 11 15:08:12.224600 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.224575 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf"] Jun 11 15:08:12.224902 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.224875 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" containerName="manager" containerID="cri-o://7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408" gracePeriod=2 Jun 11 15:08:12.232973 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.232904 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf"] Jun 11 15:08:12.242275 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.242239 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr"] Jun 11 15:08:12.275807 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:08:12.275774 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q25hf\" is forbidden: User \"system:node:ip-10-0-140-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-77.ec2.internal' and this object" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:12.278995 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.278968 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct"] Jun 11 15:08:12.279327 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.279313 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" containerName="manager" Jun 11 15:08:12.279327 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.279328 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" containerName="manager" Jun 11 15:08:12.279456 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.279352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7qk2\" (UniqueName: \"kubernetes.io/projected/daa67dfd-f78d-4341-bdc1-c44294751b0d-kube-api-access-f7qk2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9xqzr\" (UID: \"daa67dfd-f78d-4341-bdc1-c44294751b0d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:12.279456 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.279384 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" containerName="manager" Jun 11 15:08:12.279550 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.279382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa67dfd-f78d-4341-bdc1-c44294751b0d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9xqzr\" (UID: \"daa67dfd-f78d-4341-bdc1-c44294751b0d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:12.282517 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.282502 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" Jun 11 15:08:12.285197 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:08:12.285172 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q25hf\" is forbidden: User \"system:node:ip-10-0-140-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-77.ec2.internal' and this object" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:12.312068 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.311916 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct"] Jun 11 15:08:12.379975 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.379940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7qk2\" (UniqueName: \"kubernetes.io/projected/daa67dfd-f78d-4341-bdc1-c44294751b0d-kube-api-access-f7qk2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9xqzr\" (UID: \"daa67dfd-f78d-4341-bdc1-c44294751b0d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:12.380150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.379991 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa67dfd-f78d-4341-bdc1-c44294751b0d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9xqzr\" (UID: \"daa67dfd-f78d-4341-bdc1-c44294751b0d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:12.380150 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.380056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9lv5\" (UniqueName: \"kubernetes.io/projected/5d294d1e-eafb-4f15-aa24-bbd4a7194e84-kube-api-access-r9lv5\") pod \"limitador-operator-controller-manager-859d7b78bd-tfhct\" (UID: \"5d294d1e-eafb-4f15-aa24-bbd4a7194e84\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" Jun 11 15:08:12.380375 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.380355 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa67dfd-f78d-4341-bdc1-c44294751b0d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9xqzr\" (UID: \"daa67dfd-f78d-4341-bdc1-c44294751b0d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:12.388222 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.388194 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7qk2\" (UniqueName: \"kubernetes.io/projected/daa67dfd-f78d-4341-bdc1-c44294751b0d-kube-api-access-f7qk2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9xqzr\" (UID: \"daa67dfd-f78d-4341-bdc1-c44294751b0d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:12.480564 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.480483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9lv5\" (UniqueName: \"kubernetes.io/projected/5d294d1e-eafb-4f15-aa24-bbd4a7194e84-kube-api-access-r9lv5\") pod \"limitador-operator-controller-manager-859d7b78bd-tfhct\" (UID: \"5d294d1e-eafb-4f15-aa24-bbd4a7194e84\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" Jun 11 15:08:12.482846 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.482827 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:12.485164 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:08:12.485136 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q25hf\" is forbidden: User \"system:node:ip-10-0-140-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-77.ec2.internal' and this object" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:12.494975 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.494950 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9lv5\" (UniqueName: \"kubernetes.io/projected/5d294d1e-eafb-4f15-aa24-bbd4a7194e84-kube-api-access-r9lv5\") pod \"limitador-operator-controller-manager-859d7b78bd-tfhct\" (UID: \"5d294d1e-eafb-4f15-aa24-bbd4a7194e84\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" Jun 11 15:08:12.534830 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.534800 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:12.581774 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.581744 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4kvd\" (UniqueName: \"kubernetes.io/projected/3c38e1ac-8fb7-4143-b154-521c4905159e-kube-api-access-m4kvd\") pod \"3c38e1ac-8fb7-4143-b154-521c4905159e\" (UID: \"3c38e1ac-8fb7-4143-b154-521c4905159e\") " Jun 11 15:08:12.584908 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.584875 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c38e1ac-8fb7-4143-b154-521c4905159e-kube-api-access-m4kvd" (OuterVolumeSpecName: "kube-api-access-m4kvd") pod "3c38e1ac-8fb7-4143-b154-521c4905159e" (UID: "3c38e1ac-8fb7-4143-b154-521c4905159e"). InnerVolumeSpecName "kube-api-access-m4kvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:08:12.616280 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.616234 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" Jun 11 15:08:12.682789 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.682755 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4kvd\" (UniqueName: \"kubernetes.io/projected/3c38e1ac-8fb7-4143-b154-521c4905159e-kube-api-access-m4kvd\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:08:12.699036 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.699012 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr"] Jun 11 15:08:12.701680 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:08:12.701652 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa67dfd_f78d_4341_bdc1_c44294751b0d.slice/crio-d215fe63c17be2581ce48d01a7cebf41790713e1ecf9711575f4105b8e484ee7 WatchSource:0}: Error finding container d215fe63c17be2581ce48d01a7cebf41790713e1ecf9711575f4105b8e484ee7: Status 404 returned error can't find the container with id d215fe63c17be2581ce48d01a7cebf41790713e1ecf9711575f4105b8e484ee7 Jun 11 15:08:12.765191 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.765138 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct"] Jun 11 15:08:12.767325 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:08:12.767294 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d294d1e_eafb_4f15_aa24_bbd4a7194e84.slice/crio-93f5a1e289fd7d76dc58d17fbf2d84e3535b1ef0cbc89e102a6b89782b8b71af WatchSource:0}: Error finding container 93f5a1e289fd7d76dc58d17fbf2d84e3535b1ef0cbc89e102a6b89782b8b71af: Status 404 returned error can't find the container with id 93f5a1e289fd7d76dc58d17fbf2d84e3535b1ef0cbc89e102a6b89782b8b71af Jun 11 15:08:12.866060 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.866020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" event={"ID":"5d294d1e-eafb-4f15-aa24-bbd4a7194e84","Type":"ContainerStarted","Data":"5c50812f11a4cded020417484c5adac28f8c921fdb4e558d6bb5a86f6c3ec953"} Jun 11 15:08:12.866220 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.866084 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" event={"ID":"5d294d1e-eafb-4f15-aa24-bbd4a7194e84","Type":"ContainerStarted","Data":"93f5a1e289fd7d76dc58d17fbf2d84e3535b1ef0cbc89e102a6b89782b8b71af"} Jun 11 15:08:12.866220 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.866136 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" Jun 11 15:08:12.867747 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.867694 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" event={"ID":"daa67dfd-f78d-4341-bdc1-c44294751b0d","Type":"ContainerStarted","Data":"d215fe63c17be2581ce48d01a7cebf41790713e1ecf9711575f4105b8e484ee7"} Jun 11 15:08:12.868689 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:08:12.868661 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q25hf\" is forbidden: User \"system:node:ip-10-0-140-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-77.ec2.internal' and this object" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:12.869151 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.869126 2562 generic.go:356] "Generic (PLEG): container finished" podID="3c38e1ac-8fb7-4143-b154-521c4905159e" containerID="7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408" exitCode=0 Jun 11 15:08:12.869257 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.869172 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:12.869333 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.869174 2562 scope.go:117] "RemoveContainer" containerID="7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408" Jun 11 15:08:12.878781 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.878759 2562 scope.go:117] "RemoveContainer" containerID="7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408" Jun 11 15:08:12.879044 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:08:12.879023 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408\": container with ID starting with 7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408 not found: ID does not exist" containerID="7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408" Jun 11 15:08:12.879118 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.879055 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408"} err="failed to get container status \"7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408\": rpc error: code = NotFound desc = could not find container \"7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408\": container with ID starting with 7049d0e69a6086bc5fa6f1b69763568ec6c2bf11ae305cc96aecfc71c575a408 not found: ID does not exist" Jun 11 15:08:12.886066 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:12.886020 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" podStartSLOduration=0.886003486 podStartE2EDuration="886.003486ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:08:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:08:12.885123571 +0000 UTC m=+549.661755317" watchObservedRunningTime="2026-06-11 15:08:12.886003486 +0000 UTC m=+549.662635208" Jun 11 15:08:12.887313 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:08:12.887281 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q25hf\" is forbidden: User \"system:node:ip-10-0-140-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-77.ec2.internal' and this object" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:13.675683 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:08:13.675198 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q25hf\" is forbidden: User \"system:node:ip-10-0-140-77.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-77.ec2.internal' and this object" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q25hf" Jun 11 15:08:13.680977 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:13.680950 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c38e1ac-8fb7-4143-b154-521c4905159e" path="/var/lib/kubelet/pods/3c38e1ac-8fb7-4143-b154-521c4905159e/volumes" Jun 11 15:08:23.877998 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:23.877968 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tfhct" Jun 11 15:08:29.956509 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:29.956473 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" event={"ID":"daa67dfd-f78d-4341-bdc1-c44294751b0d","Type":"ContainerStarted","Data":"5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2"} Jun 11 15:08:29.957006 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:29.956606 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:29.957935 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:29.957901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" event={"ID":"e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c","Type":"ContainerStarted","Data":"d91c0e1fc3eaeabad81865607b733dfe990cc14824bc9623d5d5f5524a06618d"} Jun 11 15:08:29.978144 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:29.978096 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" podStartSLOduration=1.244982433 podStartE2EDuration="17.978083568s" totalImagesPullingTime="16.733101135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:08:29.976621752 +0000 UTC m=+566.753253472" watchObservedRunningTime="2026-06-11 15:08:29.978083568 +0000 UTC m=+566.754715289" Jun 11 15:08:29.994575 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:29.994528 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2zlx7" podStartSLOduration=1.548278413 podStartE2EDuration="28.994516025s" totalImagesPullingTime="27.446237612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:08:29.994496333 +0000 UTC m=+566.771128055" watchObservedRunningTime="2026-06-11 15:08:29.994516025 +0000 UTC m=+566.771147747" Jun 11 15:08:40.964863 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:40.964833 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:53.719469 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:53.719433 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr"] Jun 11 15:08:53.719928 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:53.719629 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" podUID="daa67dfd-f78d-4341-bdc1-c44294751b0d" containerName="manager" containerID="cri-o://5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2" gracePeriod=10 Jun 11 15:08:54.560911 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:54.560886 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:54.663371 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:54.663338 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa67dfd-f78d-4341-bdc1-c44294751b0d-extensions-socket-volume\") pod \"daa67dfd-f78d-4341-bdc1-c44294751b0d\" (UID: \"daa67dfd-f78d-4341-bdc1-c44294751b0d\") " Jun 11 15:08:54.663625 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:54.663403 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7qk2\" (UniqueName: \"kubernetes.io/projected/daa67dfd-f78d-4341-bdc1-c44294751b0d-kube-api-access-f7qk2\") pod \"daa67dfd-f78d-4341-bdc1-c44294751b0d\" (UID: \"daa67dfd-f78d-4341-bdc1-c44294751b0d\") " Jun 11 15:08:54.663864 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:54.663834 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa67dfd-f78d-4341-bdc1-c44294751b0d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "daa67dfd-f78d-4341-bdc1-c44294751b0d" (UID: "daa67dfd-f78d-4341-bdc1-c44294751b0d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 15:08:54.665477 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:54.665447 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa67dfd-f78d-4341-bdc1-c44294751b0d-kube-api-access-f7qk2" (OuterVolumeSpecName: "kube-api-access-f7qk2") pod "daa67dfd-f78d-4341-bdc1-c44294751b0d" (UID: "daa67dfd-f78d-4341-bdc1-c44294751b0d"). InnerVolumeSpecName "kube-api-access-f7qk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:08:54.764134 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:54.764050 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daa67dfd-f78d-4341-bdc1-c44294751b0d-extensions-socket-volume\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:08:54.764134 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:54.764078 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7qk2\" (UniqueName: \"kubernetes.io/projected/daa67dfd-f78d-4341-bdc1-c44294751b0d-kube-api-access-f7qk2\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:08:55.067138 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.067055 2562 generic.go:356] "Generic (PLEG): container finished" podID="daa67dfd-f78d-4341-bdc1-c44294751b0d" containerID="5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2" exitCode=0 Jun 11 15:08:55.067138 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.067102 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" Jun 11 15:08:55.067327 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.067102 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" event={"ID":"daa67dfd-f78d-4341-bdc1-c44294751b0d","Type":"ContainerDied","Data":"5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2"} Jun 11 15:08:55.067327 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.067234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr" event={"ID":"daa67dfd-f78d-4341-bdc1-c44294751b0d","Type":"ContainerDied","Data":"d215fe63c17be2581ce48d01a7cebf41790713e1ecf9711575f4105b8e484ee7"} Jun 11 15:08:55.067327 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.067267 2562 scope.go:117] "RemoveContainer" containerID="5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2" Jun 11 15:08:55.075876 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.075859 2562 scope.go:117] "RemoveContainer" containerID="5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2" Jun 11 15:08:55.076139 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:08:55.076122 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2\": container with ID starting with 5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2 not found: ID does not exist" containerID="5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2" Jun 11 15:08:55.076203 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.076153 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2"} err="failed to get container status \"5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2\": rpc error: code = NotFound desc = could not find container \"5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2\": container with ID starting with 5fb3829a5ab80686fe4d88fd68fdbd6e174a3639db288a3ee8fd4ed1f1dddee2 not found: ID does not exist" Jun 11 15:08:55.091839 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.091818 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr"] Jun 11 15:08:55.095944 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.095922 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9xqzr"] Jun 11 15:08:55.675768 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:55.675736 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daa67dfd-f78d-4341-bdc1-c44294751b0d" path="/var/lib/kubelet/pods/daa67dfd-f78d-4341-bdc1-c44294751b0d/volumes" Jun 11 15:08:57.351910 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.349454 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq"] Jun 11 15:08:57.351910 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.350265 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa67dfd-f78d-4341-bdc1-c44294751b0d" containerName="manager" Jun 11 15:08:57.351910 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.350286 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa67dfd-f78d-4341-bdc1-c44294751b0d" containerName="manager" Jun 11 15:08:57.351910 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.350431 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="daa67dfd-f78d-4341-bdc1-c44294751b0d" containerName="manager" Jun 11 15:08:57.354978 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.354948 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.358200 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.358179 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jbv7h\"" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.377775 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq"] Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v4lp\" (UniqueName: \"kubernetes.io/projected/203308f5-42e0-477e-a474-0a9e2d623408-kube-api-access-5v4lp\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394584 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/203308f5-42e0-477e-a474-0a9e2d623408-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/203308f5-42e0-477e-a474-0a9e2d623408-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/203308f5-42e0-477e-a474-0a9e2d623408-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.395734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.394795 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495285 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/203308f5-42e0-477e-a474-0a9e2d623408-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/203308f5-42e0-477e-a474-0a9e2d623408-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495335 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v4lp\" (UniqueName: \"kubernetes.io/projected/203308f5-42e0-477e-a474-0a9e2d623408-kube-api-access-5v4lp\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495485 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/203308f5-42e0-477e-a474-0a9e2d623408-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495914 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495914 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495761 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495914 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495807 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.495914 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.495867 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.496348 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.496325 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/203308f5-42e0-477e-a474-0a9e2d623408-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.497520 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.497500 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/203308f5-42e0-477e-a474-0a9e2d623408-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.497907 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.497891 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/203308f5-42e0-477e-a474-0a9e2d623408-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.504068 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.504044 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/203308f5-42e0-477e-a474-0a9e2d623408-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.504203 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.504184 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v4lp\" (UniqueName: \"kubernetes.io/projected/203308f5-42e0-477e-a474-0a9e2d623408-kube-api-access-5v4lp\") pod \"maas-default-gateway-openshift-default-687ff6996-tvsfq\" (UID: \"203308f5-42e0-477e-a474-0a9e2d623408\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.665787 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.665696 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:57.792858 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.792830 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq"] Jun 11 15:08:57.795468 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:08:57.795441 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203308f5_42e0_477e_a474_0a9e2d623408.slice/crio-d1e8af9b18079e60b226b13e465e3aca765c048dc0203197d915ab31f7feab0b WatchSource:0}: Error finding container d1e8af9b18079e60b226b13e465e3aca765c048dc0203197d915ab31f7feab0b: Status 404 returned error can't find the container with id d1e8af9b18079e60b226b13e465e3aca765c048dc0203197d915ab31f7feab0b Jun 11 15:08:57.797463 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.797428 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:08:57.797535 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.797494 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:08:57.797535 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:57.797527 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:08:58.082805 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:58.082705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" event={"ID":"203308f5-42e0-477e-a474-0a9e2d623408","Type":"ContainerStarted","Data":"af248fb27656f1ea4aa26fd55b5a81e27276d39affa1830dcae9137d6560110c"} Jun 11 15:08:58.082805 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:58.082774 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" event={"ID":"203308f5-42e0-477e-a474-0a9e2d623408","Type":"ContainerStarted","Data":"d1e8af9b18079e60b226b13e465e3aca765c048dc0203197d915ab31f7feab0b"} Jun 11 15:08:58.106386 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:58.106325 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" podStartSLOduration=1.106305133 podStartE2EDuration="1.106305133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:08:58.105799173 +0000 UTC m=+594.882430895" watchObservedRunningTime="2026-06-11 15:08:58.106305133 +0000 UTC m=+594.882936855" Jun 11 15:08:58.666426 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:58.666395 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:58.670675 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:58.670651 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:59.088124 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:59.088099 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:08:59.089042 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:08:59.089025 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:09:01.689933 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:01.689902 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vc668"] Jun 11 15:09:02.184914 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.184873 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vc668"] Jun 11 15:09:02.184914 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.184920 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dhhj"] Jun 11 15:09:02.185110 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.185036 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vc668" Jun 11 15:09:02.192747 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.192702 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5mbzz\"" Jun 11 15:09:02.206245 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.206165 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dhhj"] Jun 11 15:09:02.206245 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.206229 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9dhhj" Jun 11 15:09:02.236359 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.236326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzttm\" (UniqueName: \"kubernetes.io/projected/22710a82-e217-4480-9222-9835707cdf8a-kube-api-access-gzttm\") pod \"authorino-58cd955675-vc668\" (UID: \"22710a82-e217-4480-9222-9835707cdf8a\") " pod="kuadrant-system/authorino-58cd955675-vc668" Jun 11 15:09:02.236516 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.236391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wf9b\" (UniqueName: \"kubernetes.io/projected/64929674-8987-47d8-b4c4-2a0238608b69-kube-api-access-2wf9b\") pod \"authorino-75d556bd77-9dhhj\" (UID: \"64929674-8987-47d8-b4c4-2a0238608b69\") " pod="kuadrant-system/authorino-75d556bd77-9dhhj" Jun 11 15:09:02.337574 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.337539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzttm\" (UniqueName: \"kubernetes.io/projected/22710a82-e217-4480-9222-9835707cdf8a-kube-api-access-gzttm\") pod \"authorino-58cd955675-vc668\" (UID: \"22710a82-e217-4480-9222-9835707cdf8a\") " pod="kuadrant-system/authorino-58cd955675-vc668" Jun 11 15:09:02.337780 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.337603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wf9b\" (UniqueName: \"kubernetes.io/projected/64929674-8987-47d8-b4c4-2a0238608b69-kube-api-access-2wf9b\") pod \"authorino-75d556bd77-9dhhj\" (UID: \"64929674-8987-47d8-b4c4-2a0238608b69\") " pod="kuadrant-system/authorino-75d556bd77-9dhhj" Jun 11 15:09:02.345977 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.345953 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzttm\" (UniqueName: \"kubernetes.io/projected/22710a82-e217-4480-9222-9835707cdf8a-kube-api-access-gzttm\") pod \"authorino-58cd955675-vc668\" (UID: \"22710a82-e217-4480-9222-9835707cdf8a\") " pod="kuadrant-system/authorino-58cd955675-vc668" Jun 11 15:09:02.346124 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.346106 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wf9b\" (UniqueName: \"kubernetes.io/projected/64929674-8987-47d8-b4c4-2a0238608b69-kube-api-access-2wf9b\") pod \"authorino-75d556bd77-9dhhj\" (UID: \"64929674-8987-47d8-b4c4-2a0238608b69\") " pod="kuadrant-system/authorino-75d556bd77-9dhhj" Jun 11 15:09:02.494309 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.494230 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vc668" Jun 11 15:09:02.514434 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.514409 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9dhhj" Jun 11 15:09:02.654765 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.654735 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vc668"] Jun 11 15:09:02.657698 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:09:02.657673 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22710a82_e217_4480_9222_9835707cdf8a.slice/crio-282558f4f5f6e29bbfd7467f9cc542bfafbe6429be038e24f75569a36b7d1b9e WatchSource:0}: Error finding container 282558f4f5f6e29bbfd7467f9cc542bfafbe6429be038e24f75569a36b7d1b9e: Status 404 returned error can't find the container with id 282558f4f5f6e29bbfd7467f9cc542bfafbe6429be038e24f75569a36b7d1b9e Jun 11 15:09:02.677184 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:02.677087 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dhhj"] Jun 11 15:09:02.679236 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:09:02.679209 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64929674_8987_47d8_b4c4_2a0238608b69.slice/crio-be5004464972b10be972109d6b9b2e909624ee328dacb1b334903a0bbfec8a7d WatchSource:0}: Error finding container be5004464972b10be972109d6b9b2e909624ee328dacb1b334903a0bbfec8a7d: Status 404 returned error can't find the container with id be5004464972b10be972109d6b9b2e909624ee328dacb1b334903a0bbfec8a7d Jun 11 15:09:03.107512 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:03.107484 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vc668" event={"ID":"22710a82-e217-4480-9222-9835707cdf8a","Type":"ContainerStarted","Data":"282558f4f5f6e29bbfd7467f9cc542bfafbe6429be038e24f75569a36b7d1b9e"} Jun 11 15:09:03.108607 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:03.108583 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9dhhj" event={"ID":"64929674-8987-47d8-b4c4-2a0238608b69","Type":"ContainerStarted","Data":"be5004464972b10be972109d6b9b2e909624ee328dacb1b334903a0bbfec8a7d"} Jun 11 15:09:03.713329 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:03.713304 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 15:09:03.726832 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:03.726740 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 15:09:07.133003 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:07.132959 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vc668" event={"ID":"22710a82-e217-4480-9222-9835707cdf8a","Type":"ContainerStarted","Data":"7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a"} Jun 11 15:09:07.134270 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:07.134246 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9dhhj" event={"ID":"64929674-8987-47d8-b4c4-2a0238608b69","Type":"ContainerStarted","Data":"4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219"} Jun 11 15:09:07.153391 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:07.153340 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vc668" podStartSLOduration=2.425296238 podStartE2EDuration="6.15332562s" totalImagesPullingTime="3.728029382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:09:07.151705589 +0000 UTC m=+603.928337313" watchObservedRunningTime="2026-06-11 15:09:07.15332562 +0000 UTC m=+603.929957344" Jun 11 15:09:07.169761 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:07.169689 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9dhhj" podStartSLOduration=2.456756088 podStartE2EDuration="6.169666902s" totalImagesPullingTime="3.712910814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:09:07.168355286 +0000 UTC m=+603.944987008" watchObservedRunningTime="2026-06-11 15:09:07.169666902 +0000 UTC m=+603.946298624" Jun 11 15:09:07.197320 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:07.197283 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vc668"] Jun 11 15:09:09.142489 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:09.142452 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vc668" podUID="22710a82-e217-4480-9222-9835707cdf8a" containerName="authorino" containerID="cri-o://7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a" gracePeriod=30 Jun 11 15:09:09.386358 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:09.386332 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vc668" Jun 11 15:09:09.500904 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:09.500822 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzttm\" (UniqueName: \"kubernetes.io/projected/22710a82-e217-4480-9222-9835707cdf8a-kube-api-access-gzttm\") pod \"22710a82-e217-4480-9222-9835707cdf8a\" (UID: \"22710a82-e217-4480-9222-9835707cdf8a\") " Jun 11 15:09:09.502951 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:09.502919 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22710a82-e217-4480-9222-9835707cdf8a-kube-api-access-gzttm" (OuterVolumeSpecName: "kube-api-access-gzttm") pod "22710a82-e217-4480-9222-9835707cdf8a" (UID: "22710a82-e217-4480-9222-9835707cdf8a"). InnerVolumeSpecName "kube-api-access-gzttm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:09:09.602035 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:09.601994 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzttm\" (UniqueName: \"kubernetes.io/projected/22710a82-e217-4480-9222-9835707cdf8a-kube-api-access-gzttm\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:09:10.150144 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.150110 2562 generic.go:356] "Generic (PLEG): container finished" podID="22710a82-e217-4480-9222-9835707cdf8a" containerID="7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a" exitCode=0 Jun 11 15:09:10.150602 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.150149 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vc668" Jun 11 15:09:10.150602 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.150201 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vc668" event={"ID":"22710a82-e217-4480-9222-9835707cdf8a","Type":"ContainerDied","Data":"7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a"} Jun 11 15:09:10.150602 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.150245 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vc668" event={"ID":"22710a82-e217-4480-9222-9835707cdf8a","Type":"ContainerDied","Data":"282558f4f5f6e29bbfd7467f9cc542bfafbe6429be038e24f75569a36b7d1b9e"} Jun 11 15:09:10.150602 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.150270 2562 scope.go:117] "RemoveContainer" containerID="7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a" Jun 11 15:09:10.158177 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.158150 2562 scope.go:117] "RemoveContainer" containerID="7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a" Jun 11 15:09:10.158417 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:09:10.158401 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a\": container with ID starting with 7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a not found: ID does not exist" containerID="7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a" Jun 11 15:09:10.158475 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.158423 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a"} err="failed to get container status \"7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a\": rpc error: code = NotFound desc = could not find container \"7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a\": container with ID starting with 7ee545708b309b7d28398a43717136afc3b575273f504f6d0daebc69f0f3331a not found: ID does not exist" Jun 11 15:09:10.166445 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.166421 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vc668"] Jun 11 15:09:10.172224 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:10.172205 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vc668"] Jun 11 15:09:11.674890 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:11.674860 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22710a82-e217-4480-9222-9835707cdf8a" path="/var/lib/kubelet/pods/22710a82-e217-4480-9222-9835707cdf8a/volumes" Jun 11 15:09:55.918024 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.917989 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:09:55.919585 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.918292 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22710a82-e217-4480-9222-9835707cdf8a" containerName="authorino" Jun 11 15:09:55.919585 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.918304 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="22710a82-e217-4480-9222-9835707cdf8a" containerName="authorino" Jun 11 15:09:55.919585 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.918376 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="22710a82-e217-4480-9222-9835707cdf8a" containerName="authorino" Jun 11 15:09:55.920066 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.920052 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 15:09:55.922643 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.922622 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 11 15:09:55.922791 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.922628 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 15:09:55.923862 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.923844 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 15:09:55.923943 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.923884 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-t7jhs\"" Jun 11 15:09:55.930453 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:55.930429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:09:56.001662 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:56.001631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rv72\" (UniqueName: \"kubernetes.io/projected/0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a-kube-api-access-4rv72\") pod \"maas-keycloak-0\" (UID: \"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:09:56.102420 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:56.102390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rv72\" (UniqueName: \"kubernetes.io/projected/0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a-kube-api-access-4rv72\") pod \"maas-keycloak-0\" (UID: \"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:09:56.112539 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:56.112509 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rv72\" (UniqueName: \"kubernetes.io/projected/0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a-kube-api-access-4rv72\") pod \"maas-keycloak-0\" (UID: \"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:09:56.229863 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:56.229771 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 15:09:56.356909 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:56.356885 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:09:56.358905 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:09:56.358879 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c155644_dbc7_4bc3_a4c3_3d29b84e1d0a.slice/crio-6c16c2a566f1189d6a36551907f38c95b280f04d88f77c1723c03d952ee12718 WatchSource:0}: Error finding container 6c16c2a566f1189d6a36551907f38c95b280f04d88f77c1723c03d952ee12718: Status 404 returned error can't find the container with id 6c16c2a566f1189d6a36551907f38c95b280f04d88f77c1723c03d952ee12718 Jun 11 15:09:57.364299 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:09:57.364262 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a","Type":"ContainerStarted","Data":"6c16c2a566f1189d6a36551907f38c95b280f04d88f77c1723c03d952ee12718"} Jun 11 15:10:01.388609 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:01.388574 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a","Type":"ContainerStarted","Data":"8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59"} Jun 11 15:10:01.408545 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:01.408491 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.518452673 podStartE2EDuration="6.408473296s" totalImagesPullingTime="4.890020623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:10:01.407192544 +0000 UTC m=+658.183824269" watchObservedRunningTime="2026-06-11 15:10:01.408473296 +0000 UTC m=+658.185105013" Jun 11 15:10:02.230024 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:02.229982 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 11 15:10:02.231911 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:02.231871 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:03.230512 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:03.230446 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:04.231108 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:04.231059 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:05.230553 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:05.230491 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:06.229973 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:06.229925 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 11 15:10:06.230595 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:06.230562 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:07.230769 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:07.230669 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:08.230907 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:08.230854 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:09.230548 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:09.230508 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:10.231173 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:10.231121 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:11.230236 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:11.230189 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:12.231201 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:12.231160 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:13.231117 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:13.231073 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.35:9000/health/started\": dial tcp 10.133.0.35:9000: connect: connection refused" Jun 11 15:10:14.344553 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:14.344513 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 11 15:10:14.362629 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:14.362579 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 15:10:24.351130 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:24.351094 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 11 15:10:25.238533 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.238501 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k2v66"] Jun 11 15:10:25.246291 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.246269 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k2v66" Jun 11 15:10:25.247651 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.247624 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k2v66"] Jun 11 15:10:25.294328 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.294289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2sks\" (UniqueName: \"kubernetes.io/projected/abdb5ea4-8244-41e1-8665-67e77d1904e6-kube-api-access-k2sks\") pod \"authorino-5d4b67c446-k2v66\" (UID: \"abdb5ea4-8244-41e1-8665-67e77d1904e6\") " pod="kuadrant-system/authorino-5d4b67c446-k2v66" Jun 11 15:10:25.395036 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.395002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2sks\" (UniqueName: \"kubernetes.io/projected/abdb5ea4-8244-41e1-8665-67e77d1904e6-kube-api-access-k2sks\") pod \"authorino-5d4b67c446-k2v66\" (UID: \"abdb5ea4-8244-41e1-8665-67e77d1904e6\") " pod="kuadrant-system/authorino-5d4b67c446-k2v66" Jun 11 15:10:25.404707 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.404672 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2sks\" (UniqueName: \"kubernetes.io/projected/abdb5ea4-8244-41e1-8665-67e77d1904e6-kube-api-access-k2sks\") pod \"authorino-5d4b67c446-k2v66\" (UID: \"abdb5ea4-8244-41e1-8665-67e77d1904e6\") " pod="kuadrant-system/authorino-5d4b67c446-k2v66" Jun 11 15:10:25.466334 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.466306 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k2v66"] Jun 11 15:10:25.466551 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.466538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k2v66" Jun 11 15:10:25.497909 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.497845 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7485d6f5d9-d6zjx"] Jun 11 15:10:25.501110 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.501090 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7485d6f5d9-d6zjx" Jun 11 15:10:25.511261 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.511235 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7485d6f5d9-d6zjx"] Jun 11 15:10:25.532988 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.532957 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7485d6f5d9-d6zjx"] Jun 11 15:10:25.533258 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:10:25.533235 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pdpjn], unattached volumes=[], failed to process volumes=[kube-api-access-pdpjn]: context canceled" pod="kuadrant-system/authorino-7485d6f5d9-d6zjx" podUID="9a998480-dce9-42ea-a974-a34385074179" Jun 11 15:10:25.546081 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.546047 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7485d6f5d9-d6zjx" Jun 11 15:10:25.552474 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.552452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7485d6f5d9-d6zjx" Jun 11 15:10:25.572116 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.572024 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79765f65d5-f7xnj"] Jun 11 15:10:25.575304 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.575285 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:10:25.578022 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.578000 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 11 15:10:25.583303 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.582887 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79765f65d5-f7xnj"] Jun 11 15:10:25.596280 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.596253 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzkjg\" (UniqueName: \"kubernetes.io/projected/df7c3a99-261d-43dc-a6e1-de21344bca9b-kube-api-access-fzkjg\") pod \"authorino-79765f65d5-f7xnj\" (UID: \"df7c3a99-261d-43dc-a6e1-de21344bca9b\") " pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:10:25.596419 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.596295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df7c3a99-261d-43dc-a6e1-de21344bca9b-tls-cert\") pod \"authorino-79765f65d5-f7xnj\" (UID: \"df7c3a99-261d-43dc-a6e1-de21344bca9b\") " pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:10:25.612665 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.612645 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k2v66"] Jun 11 15:10:25.614822 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:10:25.614800 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabdb5ea4_8244_41e1_8665_67e77d1904e6.slice/crio-d49594948e7083cac5b1984369304c8b74338a560930d582e03172c6a08f967d WatchSource:0}: Error finding container d49594948e7083cac5b1984369304c8b74338a560930d582e03172c6a08f967d: Status 404 returned error can't find the container with id d49594948e7083cac5b1984369304c8b74338a560930d582e03172c6a08f967d Jun 11 15:10:25.697034 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.697006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df7c3a99-261d-43dc-a6e1-de21344bca9b-tls-cert\") pod \"authorino-79765f65d5-f7xnj\" (UID: \"df7c3a99-261d-43dc-a6e1-de21344bca9b\") " pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:10:25.697158 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.697085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzkjg\" (UniqueName: \"kubernetes.io/projected/df7c3a99-261d-43dc-a6e1-de21344bca9b-kube-api-access-fzkjg\") pod \"authorino-79765f65d5-f7xnj\" (UID: \"df7c3a99-261d-43dc-a6e1-de21344bca9b\") " pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:10:25.699424 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.699390 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df7c3a99-261d-43dc-a6e1-de21344bca9b-tls-cert\") pod \"authorino-79765f65d5-f7xnj\" (UID: \"df7c3a99-261d-43dc-a6e1-de21344bca9b\") " pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:10:25.705976 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.705959 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzkjg\" (UniqueName: \"kubernetes.io/projected/df7c3a99-261d-43dc-a6e1-de21344bca9b-kube-api-access-fzkjg\") pod \"authorino-79765f65d5-f7xnj\" (UID: \"df7c3a99-261d-43dc-a6e1-de21344bca9b\") " pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:10:25.887214 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:25.887183 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:10:26.014544 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.014520 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79765f65d5-f7xnj"] Jun 11 15:10:26.016567 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:10:26.016540 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf7c3a99_261d_43dc_a6e1_de21344bca9b.slice/crio-013d10b5b7e1c038fb280c15747376a09f2ec134be14337a3104ad188e442bd0 WatchSource:0}: Error finding container 013d10b5b7e1c038fb280c15747376a09f2ec134be14337a3104ad188e442bd0: Status 404 returned error can't find the container with id 013d10b5b7e1c038fb280c15747376a09f2ec134be14337a3104ad188e442bd0 Jun 11 15:10:26.017977 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.017961 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 15:10:26.551467 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.551369 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-k2v66" event={"ID":"abdb5ea4-8244-41e1-8665-67e77d1904e6","Type":"ContainerStarted","Data":"7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be"} Jun 11 15:10:26.551467 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.551420 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-k2v66" event={"ID":"abdb5ea4-8244-41e1-8665-67e77d1904e6","Type":"ContainerStarted","Data":"d49594948e7083cac5b1984369304c8b74338a560930d582e03172c6a08f967d"} Jun 11 15:10:26.551467 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.551405 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-k2v66" podUID="abdb5ea4-8244-41e1-8665-67e77d1904e6" containerName="authorino" containerID="cri-o://7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be" gracePeriod=30 Jun 11 15:10:26.552989 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.552972 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7485d6f5d9-d6zjx" Jun 11 15:10:26.553115 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.553001 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79765f65d5-f7xnj" event={"ID":"df7c3a99-261d-43dc-a6e1-de21344bca9b","Type":"ContainerStarted","Data":"4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5"} Jun 11 15:10:26.553115 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.553023 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79765f65d5-f7xnj" event={"ID":"df7c3a99-261d-43dc-a6e1-de21344bca9b","Type":"ContainerStarted","Data":"013d10b5b7e1c038fb280c15747376a09f2ec134be14337a3104ad188e442bd0"} Jun 11 15:10:26.568001 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.567961 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-k2v66" podStartSLOduration=0.986244747 podStartE2EDuration="1.567947305s" totalImagesPullingTime="581.702558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:10:26.566541103 +0000 UTC m=+683.343172827" watchObservedRunningTime="2026-06-11 15:10:26.567947305 +0000 UTC m=+683.344579026" Jun 11 15:10:26.591625 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.591597 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7485d6f5d9-d6zjx"] Jun 11 15:10:26.596831 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.596810 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7485d6f5d9-d6zjx"] Jun 11 15:10:26.611050 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.611011 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79765f65d5-f7xnj" podStartSLOduration=1.33271075 podStartE2EDuration="1.610997659s" totalImagesPullingTime="278.286909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:10:26.610271553 +0000 UTC m=+683.386903277" watchObservedRunningTime="2026-06-11 15:10:26.610997659 +0000 UTC m=+683.387629411" Jun 11 15:10:26.638771 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.638706 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dhhj"] Jun 11 15:10:26.638921 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.638901 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9dhhj" podUID="64929674-8987-47d8-b4c4-2a0238608b69" containerName="authorino" containerID="cri-o://4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219" gracePeriod=30 Jun 11 15:10:26.900761 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.900742 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k2v66" Jun 11 15:10:26.903531 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.903514 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9dhhj" Jun 11 15:10:26.904800 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.904782 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2sks\" (UniqueName: \"kubernetes.io/projected/abdb5ea4-8244-41e1-8665-67e77d1904e6-kube-api-access-k2sks\") pod \"abdb5ea4-8244-41e1-8665-67e77d1904e6\" (UID: \"abdb5ea4-8244-41e1-8665-67e77d1904e6\") " Jun 11 15:10:26.904863 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.904840 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wf9b\" (UniqueName: \"kubernetes.io/projected/64929674-8987-47d8-b4c4-2a0238608b69-kube-api-access-2wf9b\") pod \"64929674-8987-47d8-b4c4-2a0238608b69\" (UID: \"64929674-8987-47d8-b4c4-2a0238608b69\") " Jun 11 15:10:26.906782 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.906758 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abdb5ea4-8244-41e1-8665-67e77d1904e6-kube-api-access-k2sks" (OuterVolumeSpecName: "kube-api-access-k2sks") pod "abdb5ea4-8244-41e1-8665-67e77d1904e6" (UID: "abdb5ea4-8244-41e1-8665-67e77d1904e6"). InnerVolumeSpecName "kube-api-access-k2sks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:10:26.906862 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:26.906837 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64929674-8987-47d8-b4c4-2a0238608b69-kube-api-access-2wf9b" (OuterVolumeSpecName: "kube-api-access-2wf9b") pod "64929674-8987-47d8-b4c4-2a0238608b69" (UID: "64929674-8987-47d8-b4c4-2a0238608b69"). InnerVolumeSpecName "kube-api-access-2wf9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:10:27.005761 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.005735 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wf9b\" (UniqueName: \"kubernetes.io/projected/64929674-8987-47d8-b4c4-2a0238608b69-kube-api-access-2wf9b\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:10:27.005761 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.005758 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2sks\" (UniqueName: \"kubernetes.io/projected/abdb5ea4-8244-41e1-8665-67e77d1904e6-kube-api-access-k2sks\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:10:27.558505 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.558470 2562 generic.go:356] "Generic (PLEG): container finished" podID="64929674-8987-47d8-b4c4-2a0238608b69" containerID="4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219" exitCode=0 Jun 11 15:10:27.558933 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.558512 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9dhhj" Jun 11 15:10:27.558933 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.558561 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9dhhj" event={"ID":"64929674-8987-47d8-b4c4-2a0238608b69","Type":"ContainerDied","Data":"4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219"} Jun 11 15:10:27.558933 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.558607 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9dhhj" event={"ID":"64929674-8987-47d8-b4c4-2a0238608b69","Type":"ContainerDied","Data":"be5004464972b10be972109d6b9b2e909624ee328dacb1b334903a0bbfec8a7d"} Jun 11 15:10:27.558933 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.558634 2562 scope.go:117] "RemoveContainer" containerID="4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219" Jun 11 15:10:27.560009 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.559919 2562 generic.go:356] "Generic (PLEG): container finished" podID="abdb5ea4-8244-41e1-8665-67e77d1904e6" containerID="7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be" exitCode=0 Jun 11 15:10:27.560009 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.559959 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k2v66" Jun 11 15:10:27.560009 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.559999 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-k2v66" event={"ID":"abdb5ea4-8244-41e1-8665-67e77d1904e6","Type":"ContainerDied","Data":"7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be"} Jun 11 15:10:27.560164 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.560050 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-k2v66" event={"ID":"abdb5ea4-8244-41e1-8665-67e77d1904e6","Type":"ContainerDied","Data":"d49594948e7083cac5b1984369304c8b74338a560930d582e03172c6a08f967d"} Jun 11 15:10:27.567858 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.567845 2562 scope.go:117] "RemoveContainer" containerID="4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219" Jun 11 15:10:27.568235 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:10:27.568123 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219\": container with ID starting with 4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219 not found: ID does not exist" containerID="4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219" Jun 11 15:10:27.568235 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.568158 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219"} err="failed to get container status \"4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219\": rpc error: code = NotFound desc = could not find container \"4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219\": container with ID starting with 4f07846eedeaa57af039584163d238b8ca4a2fc70087dda1420730574aa95219 not found: ID does not exist" Jun 11 15:10:27.568235 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.568180 2562 scope.go:117] "RemoveContainer" containerID="7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be" Jun 11 15:10:27.576856 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.576838 2562 scope.go:117] "RemoveContainer" containerID="7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be" Jun 11 15:10:27.577097 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:10:27.577080 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be\": container with ID starting with 7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be not found: ID does not exist" containerID="7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be" Jun 11 15:10:27.577148 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.577103 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be"} err="failed to get container status \"7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be\": rpc error: code = NotFound desc = could not find container \"7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be\": container with ID starting with 7cf677448cac54666b9ac59792d68b3f1765fef0f79016a7c42c50a1372795be not found: ID does not exist" Jun 11 15:10:27.582547 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.582527 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k2v66"] Jun 11 15:10:27.586275 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.586256 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k2v66"] Jun 11 15:10:27.597223 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.597203 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dhhj"] Jun 11 15:10:27.603833 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.603814 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dhhj"] Jun 11 15:10:27.679055 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.678970 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64929674-8987-47d8-b4c4-2a0238608b69" path="/var/lib/kubelet/pods/64929674-8987-47d8-b4c4-2a0238608b69/volumes" Jun 11 15:10:27.679327 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.679308 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a998480-dce9-42ea-a974-a34385074179" path="/var/lib/kubelet/pods/9a998480-dce9-42ea-a974-a34385074179/volumes" Jun 11 15:10:27.679529 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:27.679513 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abdb5ea4-8244-41e1-8665-67e77d1904e6" path="/var/lib/kubelet/pods/abdb5ea4-8244-41e1-8665-67e77d1904e6/volumes" Jun 11 15:10:30.354637 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.354593 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-774f4f7676-xl97n"] Jun 11 15:10:30.355105 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.354974 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64929674-8987-47d8-b4c4-2a0238608b69" containerName="authorino" Jun 11 15:10:30.355105 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.354986 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="64929674-8987-47d8-b4c4-2a0238608b69" containerName="authorino" Jun 11 15:10:30.355105 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.355001 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abdb5ea4-8244-41e1-8665-67e77d1904e6" containerName="authorino" Jun 11 15:10:30.355105 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.355006 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="abdb5ea4-8244-41e1-8665-67e77d1904e6" containerName="authorino" Jun 11 15:10:30.355105 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.355062 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="64929674-8987-47d8-b4c4-2a0238608b69" containerName="authorino" Jun 11 15:10:30.355105 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.355074 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="abdb5ea4-8244-41e1-8665-67e77d1904e6" containerName="authorino" Jun 11 15:10:30.358159 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.358142 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:30.361116 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.361095 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bt4d5\"" Jun 11 15:10:30.361232 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.361119 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 11 15:10:30.362147 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.362132 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 11 15:10:30.370691 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.370671 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-774f4f7676-xl97n"] Jun 11 15:10:30.435599 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.435567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-webhook-cert\") pod \"maas-controller-774f4f7676-xl97n\" (UID: \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\") " pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:30.435781 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.435622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd2cj\" (UniqueName: \"kubernetes.io/projected/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-kube-api-access-qd2cj\") pod \"maas-controller-774f4f7676-xl97n\" (UID: \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\") " pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:30.536031 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.535996 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d674c68f5-9t56l"] Jun 11 15:10:30.536372 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.536350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd2cj\" (UniqueName: \"kubernetes.io/projected/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-kube-api-access-qd2cj\") pod \"maas-controller-774f4f7676-xl97n\" (UID: \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\") " pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:30.536420 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.536410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-webhook-cert\") pod \"maas-controller-774f4f7676-xl97n\" (UID: \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\") " pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:30.538490 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.538463 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:30.538786 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.538768 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-webhook-cert\") pod \"maas-controller-774f4f7676-xl97n\" (UID: \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\") " pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:30.551253 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.551226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d674c68f5-9t56l"] Jun 11 15:10:30.552394 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.552371 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd2cj\" (UniqueName: \"kubernetes.io/projected/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-kube-api-access-qd2cj\") pod \"maas-controller-774f4f7676-xl97n\" (UID: \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\") " pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:30.637227 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.637145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb77ad2d-531c-4a5c-aaee-8eb1e5a59425-webhook-cert\") pod \"maas-controller-7d674c68f5-9t56l\" (UID: \"eb77ad2d-531c-4a5c-aaee-8eb1e5a59425\") " pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:30.637371 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.637252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rggvs\" (UniqueName: \"kubernetes.io/projected/eb77ad2d-531c-4a5c-aaee-8eb1e5a59425-kube-api-access-rggvs\") pod \"maas-controller-7d674c68f5-9t56l\" (UID: \"eb77ad2d-531c-4a5c-aaee-8eb1e5a59425\") " pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:30.667798 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.667765 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:30.738011 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.737913 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rggvs\" (UniqueName: \"kubernetes.io/projected/eb77ad2d-531c-4a5c-aaee-8eb1e5a59425-kube-api-access-rggvs\") pod \"maas-controller-7d674c68f5-9t56l\" (UID: \"eb77ad2d-531c-4a5c-aaee-8eb1e5a59425\") " pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:30.738011 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.737979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb77ad2d-531c-4a5c-aaee-8eb1e5a59425-webhook-cert\") pod \"maas-controller-7d674c68f5-9t56l\" (UID: \"eb77ad2d-531c-4a5c-aaee-8eb1e5a59425\") " pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:30.740672 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.740645 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb77ad2d-531c-4a5c-aaee-8eb1e5a59425-webhook-cert\") pod \"maas-controller-7d674c68f5-9t56l\" (UID: \"eb77ad2d-531c-4a5c-aaee-8eb1e5a59425\") " pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:30.747708 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.747685 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rggvs\" (UniqueName: \"kubernetes.io/projected/eb77ad2d-531c-4a5c-aaee-8eb1e5a59425-kube-api-access-rggvs\") pod \"maas-controller-7d674c68f5-9t56l\" (UID: \"eb77ad2d-531c-4a5c-aaee-8eb1e5a59425\") " pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:30.801623 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.801596 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-774f4f7676-xl97n"] Jun 11 15:10:30.804218 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:10:30.804196 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc8088c_b3a9_4169_b5e1_7b8bb75b56f8.slice/crio-0e67bed89f0a3650e85a586148bca3051d54037d8ce4e6a7e15e51aded141727 WatchSource:0}: Error finding container 0e67bed89f0a3650e85a586148bca3051d54037d8ce4e6a7e15e51aded141727: Status 404 returned error can't find the container with id 0e67bed89f0a3650e85a586148bca3051d54037d8ce4e6a7e15e51aded141727 Jun 11 15:10:30.864352 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.864324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:30.992678 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:30.992655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d674c68f5-9t56l"] Jun 11 15:10:30.994702 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:10:30.994673 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb77ad2d_531c_4a5c_aaee_8eb1e5a59425.slice/crio-eafbdcf6fca0ded07b837028749bbdfbb73a5c85b48cd15632ea47bf4f9e72c4 WatchSource:0}: Error finding container eafbdcf6fca0ded07b837028749bbdfbb73a5c85b48cd15632ea47bf4f9e72c4: Status 404 returned error can't find the container with id eafbdcf6fca0ded07b837028749bbdfbb73a5c85b48cd15632ea47bf4f9e72c4 Jun 11 15:10:31.585109 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:31.585072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d674c68f5-9t56l" event={"ID":"eb77ad2d-531c-4a5c-aaee-8eb1e5a59425","Type":"ContainerStarted","Data":"eafbdcf6fca0ded07b837028749bbdfbb73a5c85b48cd15632ea47bf4f9e72c4"} Jun 11 15:10:31.586889 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:31.586827 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-774f4f7676-xl97n" event={"ID":"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8","Type":"ContainerStarted","Data":"0e67bed89f0a3650e85a586148bca3051d54037d8ce4e6a7e15e51aded141727"} Jun 11 15:10:33.599770 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:33.599738 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d674c68f5-9t56l" event={"ID":"eb77ad2d-531c-4a5c-aaee-8eb1e5a59425","Type":"ContainerStarted","Data":"a4754ea73d8e9ddf1910bafaa328f1c10106692f9157ae2d8e26f034b871e06d"} Jun 11 15:10:33.600211 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:33.599836 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:33.601372 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:33.601348 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-774f4f7676-xl97n" event={"ID":"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8","Type":"ContainerStarted","Data":"48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40"} Jun 11 15:10:33.601491 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:33.601482 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:33.698108 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:33.698001 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-774f4f7676-xl97n" podStartSLOduration=1.086003448 podStartE2EDuration="3.697981512s" totalImagesPullingTime="2.611978064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:10:33.696630928 +0000 UTC m=+690.473262651" watchObservedRunningTime="2026-06-11 15:10:33.697981512 +0000 UTC m=+690.474613229" Jun 11 15:10:33.702732 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:33.699483 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d674c68f5-9t56l" podStartSLOduration=1.273110081 podStartE2EDuration="3.69946339s" totalImagesPullingTime="2.426353309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:10:33.656315086 +0000 UTC m=+690.432946807" watchObservedRunningTime="2026-06-11 15:10:33.69946339 +0000 UTC m=+690.476095112" Jun 11 15:10:44.612343 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:44.612264 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d674c68f5-9t56l" Jun 11 15:10:44.612814 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:44.612350 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:44.681168 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:44.681142 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-774f4f7676-xl97n"] Jun 11 15:10:44.681376 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:44.681339 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-774f4f7676-xl97n" podUID="ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8" containerName="manager" containerID="cri-o://48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40" gracePeriod=30 Jun 11 15:10:44.933503 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:44.933480 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:45.081939 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.081909 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd2cj\" (UniqueName: \"kubernetes.io/projected/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-kube-api-access-qd2cj\") pod \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\" (UID: \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\") " Jun 11 15:10:45.082083 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.081976 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-webhook-cert\") pod \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\" (UID: \"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8\") " Jun 11 15:10:45.084167 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.084136 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8" (UID: "ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:10:45.084167 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.084157 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-kube-api-access-qd2cj" (OuterVolumeSpecName: "kube-api-access-qd2cj") pod "ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8" (UID: "ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8"). InnerVolumeSpecName "kube-api-access-qd2cj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:10:45.182923 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.182839 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd2cj\" (UniqueName: \"kubernetes.io/projected/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-kube-api-access-qd2cj\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:10:45.182923 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.182866 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8-webhook-cert\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:10:45.657510 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.657473 2562 generic.go:356] "Generic (PLEG): container finished" podID="ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8" containerID="48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40" exitCode=0 Jun 11 15:10:45.657992 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.657547 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-774f4f7676-xl97n" Jun 11 15:10:45.657992 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.657561 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-774f4f7676-xl97n" event={"ID":"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8","Type":"ContainerDied","Data":"48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40"} Jun 11 15:10:45.657992 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.657659 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-774f4f7676-xl97n" event={"ID":"ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8","Type":"ContainerDied","Data":"0e67bed89f0a3650e85a586148bca3051d54037d8ce4e6a7e15e51aded141727"} Jun 11 15:10:45.657992 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.657691 2562 scope.go:117] "RemoveContainer" containerID="48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40" Jun 11 15:10:45.667069 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.667047 2562 scope.go:117] "RemoveContainer" containerID="48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40" Jun 11 15:10:45.667331 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:10:45.667310 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40\": container with ID starting with 48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40 not found: ID does not exist" containerID="48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40" Jun 11 15:10:45.667431 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.667337 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40"} err="failed to get container status \"48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40\": rpc error: code = NotFound desc = could not find container \"48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40\": container with ID starting with 48b0b45e55194298f22cb655c77d855bca03742c87da08ec006ceb57d24c1c40 not found: ID does not exist" Jun 11 15:10:45.680590 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.680568 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-774f4f7676-xl97n"] Jun 11 15:10:45.682984 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:45.682965 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-774f4f7676-xl97n"] Jun 11 15:10:47.674257 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:10:47.674217 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8" path="/var/lib/kubelet/pods/ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8/volumes" Jun 11 15:11:04.772684 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.772653 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-845d5d9dc5-6djdw"] Jun 11 15:11:04.773118 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.773010 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8" containerName="manager" Jun 11 15:11:04.773118 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.773021 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8" containerName="manager" Jun 11 15:11:04.773118 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.773093 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc8088c-b3a9-4169-b5e1-7b8bb75b56f8" containerName="manager" Jun 11 15:11:04.778288 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.778268 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:04.782108 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.782087 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-dhfx6\"" Jun 11 15:11:04.782220 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.782092 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 11 15:11:04.791751 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.791709 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-845d5d9dc5-6djdw"] Jun 11 15:11:04.959356 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.959324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6kc4\" (UniqueName: \"kubernetes.io/projected/ff927c31-f627-441f-a647-379cea63a16a-kube-api-access-s6kc4\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:04.959548 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.959366 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff927c31-f627-441f-a647-379cea63a16a-maas-api-tls\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:04.959548 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:04.959472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff927c31-f627-441f-a647-379cea63a16a-openshift-service-ca\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:05.060402 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.060315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6kc4\" (UniqueName: \"kubernetes.io/projected/ff927c31-f627-441f-a647-379cea63a16a-kube-api-access-s6kc4\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:05.060402 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.060360 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff927c31-f627-441f-a647-379cea63a16a-maas-api-tls\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:05.060586 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.060405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff927c31-f627-441f-a647-379cea63a16a-openshift-service-ca\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:05.060991 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.060973 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff927c31-f627-441f-a647-379cea63a16a-openshift-service-ca\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:05.062889 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.062865 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff927c31-f627-441f-a647-379cea63a16a-maas-api-tls\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:05.070679 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.070660 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6kc4\" (UniqueName: \"kubernetes.io/projected/ff927c31-f627-441f-a647-379cea63a16a-kube-api-access-s6kc4\") pod \"maas-api-845d5d9dc5-6djdw\" (UID: \"ff927c31-f627-441f-a647-379cea63a16a\") " pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:05.088875 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.088853 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:05.230612 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.230504 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-845d5d9dc5-6djdw"] Jun 11 15:11:05.233117 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:11:05.233083 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff927c31_f627_441f_a647_379cea63a16a.slice/crio-4d1dbb8f16d6ab76292746d529bf663047596a921172ee2fc293737668e06e1a WatchSource:0}: Error finding container 4d1dbb8f16d6ab76292746d529bf663047596a921172ee2fc293737668e06e1a: Status 404 returned error can't find the container with id 4d1dbb8f16d6ab76292746d529bf663047596a921172ee2fc293737668e06e1a Jun 11 15:11:05.342762 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.342702 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-7xsvk"] Jun 11 15:11:05.346696 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.346678 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" Jun 11 15:11:05.350935 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.350918 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 11 15:11:05.362973 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.362952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb8zw\" (UniqueName: \"kubernetes.io/projected/e89e18ca-c9a2-44d5-9f12-5acd221e3c7e-kube-api-access-zb8zw\") pod \"payload-processing-699b6cd6c-7xsvk\" (UID: \"e89e18ca-c9a2-44d5-9f12-5acd221e3c7e\") " pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" Jun 11 15:11:05.373707 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.373683 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-7xsvk"] Jun 11 15:11:05.463602 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.463575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb8zw\" (UniqueName: \"kubernetes.io/projected/e89e18ca-c9a2-44d5-9f12-5acd221e3c7e-kube-api-access-zb8zw\") pod \"payload-processing-699b6cd6c-7xsvk\" (UID: \"e89e18ca-c9a2-44d5-9f12-5acd221e3c7e\") " pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" Jun 11 15:11:05.482229 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.482195 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb8zw\" (UniqueName: \"kubernetes.io/projected/e89e18ca-c9a2-44d5-9f12-5acd221e3c7e-kube-api-access-zb8zw\") pod \"payload-processing-699b6cd6c-7xsvk\" (UID: \"e89e18ca-c9a2-44d5-9f12-5acd221e3c7e\") " pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" Jun 11 15:11:05.656946 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.656866 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" Jun 11 15:11:05.745598 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.745562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-845d5d9dc5-6djdw" event={"ID":"ff927c31-f627-441f-a647-379cea63a16a","Type":"ContainerStarted","Data":"4d1dbb8f16d6ab76292746d529bf663047596a921172ee2fc293737668e06e1a"} Jun 11 15:11:05.785670 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:05.785645 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-7xsvk"] Jun 11 15:11:05.787626 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:11:05.787603 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode89e18ca_c9a2_44d5_9f12_5acd221e3c7e.slice/crio-e80b799557180f0e614346c73186e7ec8d533d2a7ff2f4c7e4095a93d4494a17 WatchSource:0}: Error finding container e80b799557180f0e614346c73186e7ec8d533d2a7ff2f4c7e4095a93d4494a17: Status 404 returned error can't find the container with id e80b799557180f0e614346c73186e7ec8d533d2a7ff2f4c7e4095a93d4494a17 Jun 11 15:11:06.759766 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:06.758994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" event={"ID":"e89e18ca-c9a2-44d5-9f12-5acd221e3c7e","Type":"ContainerStarted","Data":"e80b799557180f0e614346c73186e7ec8d533d2a7ff2f4c7e4095a93d4494a17"} Jun 11 15:11:07.767898 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:07.767861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-845d5d9dc5-6djdw" event={"ID":"ff927c31-f627-441f-a647-379cea63a16a","Type":"ContainerStarted","Data":"b96f633e05f15643aabf5eb620aec748bf43eb7dc003493635daca1b1d9f183f"} Jun 11 15:11:07.768353 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:07.767984 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:07.787051 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:07.786997 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-845d5d9dc5-6djdw" podStartSLOduration=1.818982264 podStartE2EDuration="3.786982746s" totalImagesPullingTime="1.968000482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:11:07.785215718 +0000 UTC m=+724.561847462" watchObservedRunningTime="2026-06-11 15:11:07.786982746 +0000 UTC m=+724.563614488" Jun 11 15:11:08.776608 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:08.776573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" event={"ID":"e89e18ca-c9a2-44d5-9f12-5acd221e3c7e","Type":"ContainerStarted","Data":"356eb7430fc20269567575e3808fb5989591d6fe4dded96b9e3af5afbf84a537"} Jun 11 15:11:08.777109 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:08.776806 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" Jun 11 15:11:08.798956 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:08.798913 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" podStartSLOduration=0.977871235 podStartE2EDuration="3.79889808s" totalImagesPullingTime="2.821026845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:11:08.796965517 +0000 UTC m=+725.573597249" watchObservedRunningTime="2026-06-11 15:11:08.79889808 +0000 UTC m=+725.575529800" Jun 11 15:11:12.797407 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:12.797376 2562 generic.go:356] "Generic (PLEG): container finished" podID="203308f5-42e0-477e-a474-0a9e2d623408" containerID="af248fb27656f1ea4aa26fd55b5a81e27276d39affa1830dcae9137d6560110c" exitCode=0 Jun 11 15:11:12.797830 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:12.797424 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" event={"ID":"203308f5-42e0-477e-a474-0a9e2d623408","Type":"ContainerDied","Data":"af248fb27656f1ea4aa26fd55b5a81e27276d39affa1830dcae9137d6560110c"} Jun 11 15:11:12.797830 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:12.797809 2562 scope.go:117] "RemoveContainer" containerID="af248fb27656f1ea4aa26fd55b5a81e27276d39affa1830dcae9137d6560110c" Jun 11 15:11:12.798803 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:12.798774 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:11:12.798877 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:12.798859 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:11:12.798926 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:12.798910 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 11 15:11:13.782646 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:13.782617 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-845d5d9dc5-6djdw" Jun 11 15:11:13.806986 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:13.806945 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" event={"ID":"203308f5-42e0-477e-a474-0a9e2d623408","Type":"ContainerStarted","Data":"7a866642026659a5cd3c87494ecb77c5ead4fdd70afd5f335eb8afc2645f94e8"} Jun 11 15:11:14.088023 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:14.087997 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:11:14.666770 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:14.666732 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:11:14.667882 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:14.667859 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-tvsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jun 11 15:11:14.667940 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:14.667907 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" podUID="203308f5-42e0-477e-a474-0a9e2d623408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jun 11 15:11:15.666832 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:15.666793 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-tvsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jun 11 15:11:15.667186 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:15.666852 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" podUID="203308f5-42e0-477e-a474-0a9e2d623408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jun 11 15:11:16.666903 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:16.666866 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-tvsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jun 11 15:11:16.667296 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:16.666926 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" podUID="203308f5-42e0-477e-a474-0a9e2d623408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jun 11 15:11:17.667031 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:17.666995 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-tvsfq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jun 11 15:11:17.667454 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:17.667050 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" podUID="203308f5-42e0-477e-a474-0a9e2d623408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jun 11 15:11:18.669898 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:18.669869 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:11:18.670844 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:18.670829 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-tvsfq" Jun 11 15:11:19.783437 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:11:19.783403 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-7xsvk" Jun 11 15:13:18.021136 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:18.021102 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:13:18.021658 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:18.021310 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" containerID="cri-o://8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59" gracePeriod=30 Jun 11 15:13:19.564164 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:19.564141 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:19.759838 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:19.759738 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rv72\" (UniqueName: \"kubernetes.io/projected/0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a-kube-api-access-4rv72\") pod \"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a\" (UID: \"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a\") " Jun 11 15:13:19.761879 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:19.761853 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a-kube-api-access-4rv72" (OuterVolumeSpecName: "kube-api-access-4rv72") pod "0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" (UID: "0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a"). InnerVolumeSpecName "kube-api-access-4rv72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:13:19.860670 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:19.860624 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rv72\" (UniqueName: \"kubernetes.io/projected/0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a-kube-api-access-4rv72\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:13:20.400378 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.400344 2562 generic.go:356] "Generic (PLEG): container finished" podID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerID="8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59" exitCode=143 Jun 11 15:13:20.400547 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.400390 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.400547 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.400427 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a","Type":"ContainerDied","Data":"8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59"} Jun 11 15:13:20.400547 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.400479 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a","Type":"ContainerDied","Data":"6c16c2a566f1189d6a36551907f38c95b280f04d88f77c1723c03d952ee12718"} Jun 11 15:13:20.400547 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.400505 2562 scope.go:117] "RemoveContainer" containerID="8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59" Jun 11 15:13:20.412242 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.412221 2562 scope.go:117] "RemoveContainer" containerID="8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59" Jun 11 15:13:20.412529 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:13:20.412507 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59\": container with ID starting with 8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59 not found: ID does not exist" containerID="8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59" Jun 11 15:13:20.412576 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.412542 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59"} err="failed to get container status \"8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59\": rpc error: code = NotFound desc = could not find container \"8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59\": container with ID starting with 8953916c440c83485aad02087692cf6ea032335340487222af0b6b0e3e541d59 not found: ID does not exist" Jun 11 15:13:20.425673 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.425650 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:13:20.431165 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.431143 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:13:20.459552 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.459525 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:13:20.459919 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.459905 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" Jun 11 15:13:20.459976 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.459921 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" Jun 11 15:13:20.460012 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.459992 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" containerName="keycloak" Jun 11 15:13:20.462975 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.462959 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.464683 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.464658 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/dde39ee5-28c9-4584-9eb3-f99ab00a52c5-test-realms\") pod \"maas-keycloak-0\" (UID: \"dde39ee5-28c9-4584-9eb3-f99ab00a52c5\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.464779 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.464705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbvlw\" (UniqueName: \"kubernetes.io/projected/dde39ee5-28c9-4584-9eb3-f99ab00a52c5-kube-api-access-xbvlw\") pod \"maas-keycloak-0\" (UID: \"dde39ee5-28c9-4584-9eb3-f99ab00a52c5\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.465991 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.465962 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-t7jhs\"" Jun 11 15:13:20.465991 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.465979 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 11 15:13:20.466138 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.465986 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 15:13:20.466328 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.466310 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 11 15:13:20.466403 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.466332 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 15:13:20.476379 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.476359 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:13:20.565232 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.565204 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/dde39ee5-28c9-4584-9eb3-f99ab00a52c5-test-realms\") pod \"maas-keycloak-0\" (UID: \"dde39ee5-28c9-4584-9eb3-f99ab00a52c5\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.565618 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.565238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbvlw\" (UniqueName: \"kubernetes.io/projected/dde39ee5-28c9-4584-9eb3-f99ab00a52c5-kube-api-access-xbvlw\") pod \"maas-keycloak-0\" (UID: \"dde39ee5-28c9-4584-9eb3-f99ab00a52c5\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.565920 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.565902 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/dde39ee5-28c9-4584-9eb3-f99ab00a52c5-test-realms\") pod \"maas-keycloak-0\" (UID: \"dde39ee5-28c9-4584-9eb3-f99ab00a52c5\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.575008 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.574986 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbvlw\" (UniqueName: \"kubernetes.io/projected/dde39ee5-28c9-4584-9eb3-f99ab00a52c5-kube-api-access-xbvlw\") pod \"maas-keycloak-0\" (UID: \"dde39ee5-28c9-4584-9eb3-f99ab00a52c5\") " pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.772310 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.772228 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:20.906452 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:20.906429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 15:13:20.908365 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:13:20.908339 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde39ee5_28c9_4584_9eb3_f99ab00a52c5.slice/crio-77edfff611e0b1cd497c10206c8d69b6b766177cc4275441c611b0be1813a76a WatchSource:0}: Error finding container 77edfff611e0b1cd497c10206c8d69b6b766177cc4275441c611b0be1813a76a: Status 404 returned error can't find the container with id 77edfff611e0b1cd497c10206c8d69b6b766177cc4275441c611b0be1813a76a Jun 11 15:13:21.408413 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:21.408380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"dde39ee5-28c9-4584-9eb3-f99ab00a52c5","Type":"ContainerStarted","Data":"c02b621025c37f3eba2e3866db0017b4a1336f107d251bcc46f8e7fb03fa2d48"} Jun 11 15:13:21.408574 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:21.408433 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"dde39ee5-28c9-4584-9eb3-f99ab00a52c5","Type":"ContainerStarted","Data":"77edfff611e0b1cd497c10206c8d69b6b766177cc4275441c611b0be1813a76a"} Jun 11 15:13:21.431491 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:21.431427 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.146800359 podStartE2EDuration="1.431409155s" totalImagesPullingTime="284.608796ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:13:21.430691666 +0000 UTC m=+858.207323388" watchObservedRunningTime="2026-06-11 15:13:21.431409155 +0000 UTC m=+858.208040876" Jun 11 15:13:21.677439 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:21.677406 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a" path="/var/lib/kubelet/pods/0c155644-dbc7-4bc3-a4c3-3d29b84e1d0a/volumes" Jun 11 15:13:21.773164 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:21.773123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:21.774763 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:21.774728 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:22.772666 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:22.772616 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:23.773246 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:23.773204 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:24.772877 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:24.772822 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:25.773248 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:25.773200 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:26.773338 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:26.773291 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:27.773117 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:27.772645 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:28.773118 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:28.773065 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:29.772995 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:29.772947 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:30.773052 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:30.773017 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:30.773525 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:30.773286 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:31.773423 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:31.773369 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:32.773078 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:32.773026 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:33.772587 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:33.772549 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.43:9000/health/started\": dial tcp 10.133.0.43:9000: connect: connection refused" Jun 11 15:13:34.901960 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:34.901733 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:34.921088 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:34.921028 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="dde39ee5-28c9-4584-9eb3-f99ab00a52c5" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 15:13:44.908086 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:44.907992 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 11 15:13:55.657112 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.657079 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-658bf8c56f-c76nr"] Jun 11 15:13:55.660354 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.660337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.663422 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.663402 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 11 15:13:55.677225 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.677204 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-658bf8c56f-c76nr"] Jun 11 15:13:55.812223 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.812189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/2bccfb08-b8b0-4eca-ac07-02432f452315-oidc-ca\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.812399 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.812324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g4cm\" (UniqueName: \"kubernetes.io/projected/2bccfb08-b8b0-4eca-ac07-02432f452315-kube-api-access-4g4cm\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.812399 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.812378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2bccfb08-b8b0-4eca-ac07-02432f452315-tls-cert\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.913946 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.913853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2bccfb08-b8b0-4eca-ac07-02432f452315-tls-cert\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.913946 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.913934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/2bccfb08-b8b0-4eca-ac07-02432f452315-oidc-ca\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.914143 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.913978 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g4cm\" (UniqueName: \"kubernetes.io/projected/2bccfb08-b8b0-4eca-ac07-02432f452315-kube-api-access-4g4cm\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.914587 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.914563 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/2bccfb08-b8b0-4eca-ac07-02432f452315-oidc-ca\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.916243 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.916222 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2bccfb08-b8b0-4eca-ac07-02432f452315-tls-cert\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.921974 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.921950 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g4cm\" (UniqueName: \"kubernetes.io/projected/2bccfb08-b8b0-4eca-ac07-02432f452315-kube-api-access-4g4cm\") pod \"authorino-658bf8c56f-c76nr\" (UID: \"2bccfb08-b8b0-4eca-ac07-02432f452315\") " pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:55.969643 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:55.969616 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-658bf8c56f-c76nr" Jun 11 15:13:56.099297 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:56.099272 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-658bf8c56f-c76nr"] Jun 11 15:13:56.101986 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:13:56.101956 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bccfb08_b8b0_4eca_ac07_02432f452315.slice/crio-d5d7dfadb4a3f4243edd73b5d7a7cd98c2d884486fe95db6af3c7559d37e46bc WatchSource:0}: Error finding container d5d7dfadb4a3f4243edd73b5d7a7cd98c2d884486fe95db6af3c7559d37e46bc: Status 404 returned error can't find the container with id d5d7dfadb4a3f4243edd73b5d7a7cd98c2d884486fe95db6af3c7559d37e46bc Jun 11 15:13:56.606941 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:56.606908 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-658bf8c56f-c76nr" event={"ID":"2bccfb08-b8b0-4eca-ac07-02432f452315","Type":"ContainerStarted","Data":"c87ab9fa88761ee1d19c0b1d4e42a76b8f2b7e77433a8f94cdf97e05d8fee96b"} Jun 11 15:13:56.607112 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:56.606951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-658bf8c56f-c76nr" event={"ID":"2bccfb08-b8b0-4eca-ac07-02432f452315","Type":"ContainerStarted","Data":"d5d7dfadb4a3f4243edd73b5d7a7cd98c2d884486fe95db6af3c7559d37e46bc"} Jun 11 15:13:56.624152 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:56.624096 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-658bf8c56f-c76nr" podStartSLOduration=1.304792694 podStartE2EDuration="1.624079383s" totalImagesPullingTime="319.286689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:13:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:13:56.623145626 +0000 UTC m=+893.399777360" watchObservedRunningTime="2026-06-11 15:13:56.624079383 +0000 UTC m=+893.400711104" Jun 11 15:13:56.649181 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:56.649147 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79765f65d5-f7xnj"] Jun 11 15:13:56.649338 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:56.649302 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-79765f65d5-f7xnj" podUID="df7c3a99-261d-43dc-a6e1-de21344bca9b" containerName="authorino" containerID="cri-o://4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5" gracePeriod=30 Jun 11 15:13:56.891563 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:56.891539 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:13:57.024408 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.024323 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzkjg\" (UniqueName: \"kubernetes.io/projected/df7c3a99-261d-43dc-a6e1-de21344bca9b-kube-api-access-fzkjg\") pod \"df7c3a99-261d-43dc-a6e1-de21344bca9b\" (UID: \"df7c3a99-261d-43dc-a6e1-de21344bca9b\") " Jun 11 15:13:57.024408 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.024381 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df7c3a99-261d-43dc-a6e1-de21344bca9b-tls-cert\") pod \"df7c3a99-261d-43dc-a6e1-de21344bca9b\" (UID: \"df7c3a99-261d-43dc-a6e1-de21344bca9b\") " Jun 11 15:13:57.026400 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.026362 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df7c3a99-261d-43dc-a6e1-de21344bca9b-kube-api-access-fzkjg" (OuterVolumeSpecName: "kube-api-access-fzkjg") pod "df7c3a99-261d-43dc-a6e1-de21344bca9b" (UID: "df7c3a99-261d-43dc-a6e1-de21344bca9b"). InnerVolumeSpecName "kube-api-access-fzkjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 15:13:57.034429 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.034400 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df7c3a99-261d-43dc-a6e1-de21344bca9b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "df7c3a99-261d-43dc-a6e1-de21344bca9b" (UID: "df7c3a99-261d-43dc-a6e1-de21344bca9b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 15:13:57.125237 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.125209 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzkjg\" (UniqueName: \"kubernetes.io/projected/df7c3a99-261d-43dc-a6e1-de21344bca9b-kube-api-access-fzkjg\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:13:57.125237 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.125233 2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/df7c3a99-261d-43dc-a6e1-de21344bca9b-tls-cert\") on node \"ip-10-0-140-77.ec2.internal\" DevicePath \"\"" Jun 11 15:13:57.613276 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.613243 2562 generic.go:356] "Generic (PLEG): container finished" podID="df7c3a99-261d-43dc-a6e1-de21344bca9b" containerID="4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5" exitCode=0 Jun 11 15:13:57.613449 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.613288 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79765f65d5-f7xnj" Jun 11 15:13:57.613449 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.613328 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79765f65d5-f7xnj" event={"ID":"df7c3a99-261d-43dc-a6e1-de21344bca9b","Type":"ContainerDied","Data":"4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5"} Jun 11 15:13:57.613449 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.613367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79765f65d5-f7xnj" event={"ID":"df7c3a99-261d-43dc-a6e1-de21344bca9b","Type":"ContainerDied","Data":"013d10b5b7e1c038fb280c15747376a09f2ec134be14337a3104ad188e442bd0"} Jun 11 15:13:57.613449 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.613385 2562 scope.go:117] "RemoveContainer" containerID="4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5" Jun 11 15:13:57.622402 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.622385 2562 scope.go:117] "RemoveContainer" containerID="4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5" Jun 11 15:13:57.622638 ip-10-0-140-77 kubenswrapper[2562]: E0611 15:13:57.622619 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5\": container with ID starting with 4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5 not found: ID does not exist" containerID="4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5" Jun 11 15:13:57.622691 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.622646 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5"} err="failed to get container status \"4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5\": rpc error: code = NotFound desc = could not find container \"4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5\": container with ID starting with 4ac6a7fd402634d0509157f6feb1d8c52ececc29d121fd7fcb898ef0b38115c5 not found: ID does not exist" Jun 11 15:13:57.633851 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.633826 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79765f65d5-f7xnj"] Jun 11 15:13:57.639584 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.639564 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79765f65d5-f7xnj"] Jun 11 15:13:57.675322 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:13:57.675295 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df7c3a99-261d-43dc-a6e1-de21344bca9b" path="/var/lib/kubelet/pods/df7c3a99-261d-43dc-a6e1-de21344bca9b/volumes" Jun 11 15:14:03.747463 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:03.747437 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 15:14:03.765667 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:03.765647 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 15:14:13.339641 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.339603 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n"] Jun 11 15:14:13.340182 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.340165 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df7c3a99-261d-43dc-a6e1-de21344bca9b" containerName="authorino" Jun 11 15:14:13.340229 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.340185 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="df7c3a99-261d-43dc-a6e1-de21344bca9b" containerName="authorino" Jun 11 15:14:13.340286 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.340275 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="df7c3a99-261d-43dc-a6e1-de21344bca9b" containerName="authorino" Jun 11 15:14:13.345527 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.345506 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.348429 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.348403 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 11 15:14:13.348429 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.348422 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 11 15:14:13.348604 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.348494 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 11 15:14:13.349638 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.349624 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4rhpz\"" Jun 11 15:14:13.353729 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.353679 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n"] Jun 11 15:14:13.377574 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.377541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.377747 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.377581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.377747 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.377609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.377747 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.377683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.377916 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.377746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e81f5626-fd32-49e3-bcf2-545da87ae126-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.377916 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.377772 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df8mw\" (UniqueName: \"kubernetes.io/projected/e81f5626-fd32-49e3-bcf2-545da87ae126-kube-api-access-df8mw\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.478600 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.478561 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e81f5626-fd32-49e3-bcf2-545da87ae126-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.478600 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.478601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df8mw\" (UniqueName: \"kubernetes.io/projected/e81f5626-fd32-49e3-bcf2-545da87ae126-kube-api-access-df8mw\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.478827 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.478668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.478827 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.478737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.478827 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.478780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.478827 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.478822 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.479195 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.479172 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.479273 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.479226 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.479273 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.479228 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.481009 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.480982 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e81f5626-fd32-49e3-bcf2-545da87ae126-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.481301 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.481282 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e81f5626-fd32-49e3-bcf2-545da87ae126-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.487304 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.487283 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df8mw\" (UniqueName: \"kubernetes.io/projected/e81f5626-fd32-49e3-bcf2-545da87ae126-kube-api-access-df8mw\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-r269n\" (UID: \"e81f5626-fd32-49e3-bcf2-545da87ae126\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.657738 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.657628 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:13.797483 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:13.797452 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n"] Jun 11 15:14:13.799533 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:14:13.799498 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81f5626_fd32_49e3_bcf2_545da87ae126.slice/crio-40394d8a5f76118a0b578f1df49a9ea1bf3e02bfa56637492eb023ceba668966 WatchSource:0}: Error finding container 40394d8a5f76118a0b578f1df49a9ea1bf3e02bfa56637492eb023ceba668966: Status 404 returned error can't find the container with id 40394d8a5f76118a0b578f1df49a9ea1bf3e02bfa56637492eb023ceba668966 Jun 11 15:14:14.700461 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:14.700417 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" event={"ID":"e81f5626-fd32-49e3-bcf2-545da87ae126","Type":"ContainerStarted","Data":"40394d8a5f76118a0b578f1df49a9ea1bf3e02bfa56637492eb023ceba668966"} Jun 11 15:14:19.725857 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:19.725822 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" event={"ID":"e81f5626-fd32-49e3-bcf2-545da87ae126","Type":"ContainerStarted","Data":"2981989a4672cec6ee671c9c851277106a280083cf5f782cab25b3f1ab7f578e"} Jun 11 15:14:21.737056 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:21.737023 2562 generic.go:356] "Generic (PLEG): container finished" podID="e81f5626-fd32-49e3-bcf2-545da87ae126" containerID="2981989a4672cec6ee671c9c851277106a280083cf5f782cab25b3f1ab7f578e" exitCode=0 Jun 11 15:14:21.737434 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:21.737071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" event={"ID":"e81f5626-fd32-49e3-bcf2-545da87ae126","Type":"ContainerDied","Data":"2981989a4672cec6ee671c9c851277106a280083cf5f782cab25b3f1ab7f578e"} Jun 11 15:14:24.752682 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:24.752651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" event={"ID":"e81f5626-fd32-49e3-bcf2-545da87ae126","Type":"ContainerStarted","Data":"726c10fa4f1b58457612651d6daa482d0f292d11a25a06d35fbc98f0e02aefd0"} Jun 11 15:14:24.772447 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:24.772403 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" podStartSLOduration=2.456178794 podStartE2EDuration="11.772387376s" totalImagesPullingTime="7.327524442s" totalInitContainerRuntime="1.98868414s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:14:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:14:24.770924897 +0000 UTC m=+921.547556630" watchObservedRunningTime="2026-06-11 15:14:24.772387376 +0000 UTC m=+921.549019097" Jun 11 15:14:30.612442 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.612409 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t"] Jun 11 15:14:30.617304 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.617278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.620073 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.620052 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 11 15:14:30.628867 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.628845 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t"] Jun 11 15:14:30.743820 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.743786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.744041 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.743853 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.744041 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.743888 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvbfv\" (UniqueName: \"kubernetes.io/projected/5ed0df61-9c23-4536-adad-88bd9aa63409-kube-api-access-mvbfv\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.744041 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.743950 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.744041 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.743989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ed0df61-9c23-4536-adad-88bd9aa63409-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.744041 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.744019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.844749 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.844709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.844910 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.844760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvbfv\" (UniqueName: \"kubernetes.io/projected/5ed0df61-9c23-4536-adad-88bd9aa63409-kube-api-access-mvbfv\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.844910 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.844820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.844910 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.844851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ed0df61-9c23-4536-adad-88bd9aa63409-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.844910 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.844884 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.845112 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.844934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.845163 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.845139 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.845224 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.845192 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.845326 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.845305 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.847021 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.846997 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ed0df61-9c23-4536-adad-88bd9aa63409-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.847317 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.847298 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ed0df61-9c23-4536-adad-88bd9aa63409-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.854078 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.854050 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvbfv\" (UniqueName: \"kubernetes.io/projected/5ed0df61-9c23-4536-adad-88bd9aa63409-kube-api-access-mvbfv\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t\" (UID: \"5ed0df61-9c23-4536-adad-88bd9aa63409\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:30.928038 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:30.927957 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:31.059764 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:31.059739 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t"] Jun 11 15:14:31.062339 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:14:31.062313 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed0df61_9c23_4536_adad_88bd9aa63409.slice/crio-94f6d69152831c54b3825d8e98fb96249b905cf2d77b29a9da81e84d16129887 WatchSource:0}: Error finding container 94f6d69152831c54b3825d8e98fb96249b905cf2d77b29a9da81e84d16129887: Status 404 returned error can't find the container with id 94f6d69152831c54b3825d8e98fb96249b905cf2d77b29a9da81e84d16129887 Jun 11 15:14:31.787899 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:31.787861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" event={"ID":"5ed0df61-9c23-4536-adad-88bd9aa63409","Type":"ContainerStarted","Data":"7beb701841dac2a1571b7d75de1466127a6c2380ea3f4f942ccdc981eea3d952"} Jun 11 15:14:31.788383 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:31.787914 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" event={"ID":"5ed0df61-9c23-4536-adad-88bd9aa63409","Type":"ContainerStarted","Data":"94f6d69152831c54b3825d8e98fb96249b905cf2d77b29a9da81e84d16129887"} Jun 11 15:14:32.793817 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:32.793707 2562 generic.go:356] "Generic (PLEG): container finished" podID="5ed0df61-9c23-4536-adad-88bd9aa63409" containerID="7beb701841dac2a1571b7d75de1466127a6c2380ea3f4f942ccdc981eea3d952" exitCode=0 Jun 11 15:14:32.794179 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:32.793819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" event={"ID":"5ed0df61-9c23-4536-adad-88bd9aa63409","Type":"ContainerDied","Data":"7beb701841dac2a1571b7d75de1466127a6c2380ea3f4f942ccdc981eea3d952"} Jun 11 15:14:33.658110 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:33.658056 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:33.658296 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:33.658147 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:33.680475 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:33.680445 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:33.799599 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:33.799557 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" event={"ID":"5ed0df61-9c23-4536-adad-88bd9aa63409","Type":"ContainerStarted","Data":"7e22453429bc3fbe8fbdee4c1390abebc7f62b20af0ee92f8ef5898297941a3b"} Jun 11 15:14:33.810708 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:33.810688 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-r269n" Jun 11 15:14:33.819364 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:33.819286 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" podStartSLOduration=2.243159102 podStartE2EDuration="3.819269677s" totalImagesPullingTime="219.887475ms" totalInitContainerRuntime="1.3562231s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:14:33.818701131 +0000 UTC m=+930.595332852" watchObservedRunningTime="2026-06-11 15:14:33.819269677 +0000 UTC m=+930.595901399" Jun 11 15:14:40.928859 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:40.928825 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:40.929280 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:40.928883 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:40.941461 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:40.941434 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:41.850273 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:41.850241 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t" Jun 11 15:14:45.639972 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.639940 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk"] Jun 11 15:14:45.644093 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.644075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.647899 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.647882 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 11 15:14:45.658506 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.658481 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk"] Jun 11 15:14:45.677228 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.677201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.677339 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.677250 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.677339 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.677305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d354704-8045-44aa-8137-bc5d55077b22-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.677339 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.677325 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.677446 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.677356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.677446 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.677387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rvwz\" (UniqueName: \"kubernetes.io/projected/2d354704-8045-44aa-8137-bc5d55077b22-kube-api-access-2rvwz\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.777902 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.777867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.778072 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.777921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.778072 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.777954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d354704-8045-44aa-8137-bc5d55077b22-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.778072 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.777976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.778072 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.777998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.778072 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.778034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rvwz\" (UniqueName: \"kubernetes.io/projected/2d354704-8045-44aa-8137-bc5d55077b22-kube-api-access-2rvwz\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.778334 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.778293 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.778414 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.778373 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.778469 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.778417 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.780223 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.780206 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d354704-8045-44aa-8137-bc5d55077b22-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.780502 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.780481 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d354704-8045-44aa-8137-bc5d55077b22-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.799917 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.799896 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rvwz\" (UniqueName: \"kubernetes.io/projected/2d354704-8045-44aa-8137-bc5d55077b22-kube-api-access-2rvwz\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk\" (UID: \"2d354704-8045-44aa-8137-bc5d55077b22\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:45.954702 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:45.954618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:14:46.088024 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:46.087996 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk"] Jun 11 15:14:46.091163 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:14:46.091140 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d354704_8045_44aa_8137_bc5d55077b22.slice/crio-af04243acb1095b4284962b1f026c57acfe43df3b7a49546285325224df6a971 WatchSource:0}: Error finding container af04243acb1095b4284962b1f026c57acfe43df3b7a49546285325224df6a971: Status 404 returned error can't find the container with id af04243acb1095b4284962b1f026c57acfe43df3b7a49546285325224df6a971 Jun 11 15:14:46.864352 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:46.864311 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" event={"ID":"2d354704-8045-44aa-8137-bc5d55077b22","Type":"ContainerStarted","Data":"601e7f3193111f08002363607a6cf9f41969ed33c277e2741005ff98107f3ab2"} Jun 11 15:14:46.864872 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:46.864362 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" event={"ID":"2d354704-8045-44aa-8137-bc5d55077b22","Type":"ContainerStarted","Data":"af04243acb1095b4284962b1f026c57acfe43df3b7a49546285325224df6a971"} Jun 11 15:14:51.530494 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.530461 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd"] Jun 11 15:14:51.548072 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.548039 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd"] Jun 11 15:14:51.548225 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.548138 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.550976 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.550952 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 11 15:14:51.629581 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.629550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.629581 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.629581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnv4j\" (UniqueName: \"kubernetes.io/projected/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-kube-api-access-pnv4j\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.629842 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.629617 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.629842 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.629700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.629842 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.629776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.629842 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.629812 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.730699 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.730670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.730699 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.730705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnv4j\" (UniqueName: \"kubernetes.io/projected/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-kube-api-access-pnv4j\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.730970 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.730898 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.730970 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.730962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.731062 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.731006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.731062 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.731054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.731501 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.731432 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.731629 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.731497 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.731629 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.731525 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.733481 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.733460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.733763 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.733736 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.739272 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.739255 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnv4j\" (UniqueName: \"kubernetes.io/projected/f78d8c8c-1caa-4a1e-a2d1-15879a3e5693-kube-api-access-pnv4j\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-dhppd\" (UID: \"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.857886 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:51.857859 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:14:51.999840 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:14:51.999802 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78d8c8c_1caa_4a1e_a2d1_15879a3e5693.slice/crio-6f736f0934d1dddb83340c20462ae823d9d3370e54dcd672d3ff922a5fdc4784 WatchSource:0}: Error finding container 6f736f0934d1dddb83340c20462ae823d9d3370e54dcd672d3ff922a5fdc4784: Status 404 returned error can't find the container with id 6f736f0934d1dddb83340c20462ae823d9d3370e54dcd672d3ff922a5fdc4784 Jun 11 15:14:52.000587 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:52.000567 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd"] Jun 11 15:14:52.894314 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:52.894283 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" event={"ID":"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693","Type":"ContainerStarted","Data":"de836c897abe5c4773e1b24bbe279bbe505eac2f0c8b8c0534e891a6a528e385"} Jun 11 15:14:52.894728 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:52.894324 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" event={"ID":"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693","Type":"ContainerStarted","Data":"6f736f0934d1dddb83340c20462ae823d9d3370e54dcd672d3ff922a5fdc4784"} Jun 11 15:14:54.915804 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:54.915767 2562 generic.go:356] "Generic (PLEG): container finished" podID="f78d8c8c-1caa-4a1e-a2d1-15879a3e5693" containerID="de836c897abe5c4773e1b24bbe279bbe505eac2f0c8b8c0534e891a6a528e385" exitCode=0 Jun 11 15:14:54.916187 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:54.915843 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" event={"ID":"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693","Type":"ContainerDied","Data":"de836c897abe5c4773e1b24bbe279bbe505eac2f0c8b8c0534e891a6a528e385"} Jun 11 15:14:55.922726 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:55.922681 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" event={"ID":"f78d8c8c-1caa-4a1e-a2d1-15879a3e5693","Type":"ContainerStarted","Data":"7978050c8cc03bca325a6e6658bf6167864a115f4920048e4ca1d6026c1662b0"} Jun 11 15:14:55.944723 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:14:55.944669 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" podStartSLOduration=2.327446862 podStartE2EDuration="4.944643985s" totalImagesPullingTime="302.181372ms" totalInitContainerRuntime="2.315015751s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:14:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:14:55.942604907 +0000 UTC m=+952.719236655" watchObservedRunningTime="2026-06-11 15:14:55.944643985 +0000 UTC m=+952.721275706" Jun 11 15:15:01.858200 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:01.858167 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:15:01.858669 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:01.858239 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:15:01.871578 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:01.871549 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:15:01.964915 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:01.964888 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-dhppd" Jun 11 15:15:43.155062 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:43.155027 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d354704-8045-44aa-8137-bc5d55077b22" containerID="601e7f3193111f08002363607a6cf9f41969ed33c277e2741005ff98107f3ab2" exitCode=0 Jun 11 15:15:43.155062 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:43.155063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" event={"ID":"2d354704-8045-44aa-8137-bc5d55077b22","Type":"ContainerDied","Data":"601e7f3193111f08002363607a6cf9f41969ed33c277e2741005ff98107f3ab2"} Jun 11 15:15:43.155679 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:43.155664 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 15:15:44.162163 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:44.162128 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" event={"ID":"2d354704-8045-44aa-8137-bc5d55077b22","Type":"ContainerStarted","Data":"fe8cf517556ccdb98e0b00e4e6d6a4c901e622b107df6c3fad751979a4ad90fa"} Jun 11 15:15:44.183488 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:44.183441 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" podStartSLOduration=2.408032991 podStartE2EDuration="59.183426663s" totalImagesPullingTime="322.012357ms" totalInitContainerRuntime="56.453381315s" isStatefulPod=false podCreationTimestamp="2026-06-11 15:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:15:44.180858497 +0000 UTC m=+1000.957490218" watchObservedRunningTime="2026-06-11 15:15:44.183426663 +0000 UTC m=+1000.960058383" Jun 11 15:15:45.955032 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:45.954997 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:15:45.955380 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:45.955063 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:15:45.967618 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:45.967594 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:15:46.183013 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:15:46.182983 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk" Jun 11 15:19:03.775706 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:19:03.775677 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 15:19:03.795527 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:19:03.795503 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h9k9f_945a32d3-e116-4317-b5c7-23856c9645fb/ovn-acl-logging/0.log" Jun 11 15:21:02.049283 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:02.049252 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-658bf8c56f-c76nr_2bccfb08-b8b0-4eca-ac07-02432f452315/authorino/0.log" Jun 11 15:21:06.815993 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:06.815962 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-74d6667c75-8hgrn_f984a2c7-2a44-4d0b-ab25-65db6fb3a47d/nginx/0.log" Jun 11 15:21:07.180397 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:07.180333 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-845d5d9dc5-6djdw_ff927c31-f627-441f-a647-379cea63a16a/maas-api/0.log" Jun 11 15:21:07.420563 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:07.420537 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d674c68f5-9t56l_eb77ad2d-531c-4a5c-aaee-8eb1e5a59425/manager/0.log" Jun 11 15:21:07.803388 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:07.803321 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh_63aa2c07-839a-4e40-8dfe-06bb7782045e/manager/0.log" Jun 11 15:21:07.811611 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:07.811513 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh_63aa2c07-839a-4e40-8dfe-06bb7782045e/copy-manifests/0.log" Jun 11 15:21:09.416136 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:09.416110 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-658bf8c56f-c76nr_2bccfb08-b8b0-4eca-ac07-02432f452315/authorino/0.log" Jun 11 15:21:09.664835 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:09.664801 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-n59rn_6d35b04a-0e6d-45d0-82c0-189b14a9cba3/manager/0.log" Jun 11 15:21:09.772208 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:09.772127 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2zlx7_e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c/kuadrant-console-plugin/0.log" Jun 11 15:21:09.884218 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:09.884168 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h7t6t_a0a2f432-9536-4e09-9aea-6fdc24fdc356/registry-server/0.log" Jun 11 15:21:10.215976 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:10.215936 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tfhct_5d294d1e-eafb-4f15-aa24-bbd4a7194e84/manager/0.log" Jun 11 15:21:10.589527 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:10.589505 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795sl7jz_44715b9a-f62e-4b39-b48d-4dd4c0b761a1/istio-proxy/0.log" Jun 11 15:21:11.083735 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:11.083620 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/1.log" Jun 11 15:21:11.307309 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:11.307279 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-7xsvk_e89e18ca-c9a2-44d5-9f12-5acd221e3c7e/payload-processing/0.log" Jun 11 15:21:11.417055 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:11.417029 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-868f598b87-7svzh_5c39500a-f1fc-4a5f-b922-1ff7217234b0/router/0.log" Jun 11 15:21:11.761321 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:11.761219 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-r269n_e81f5626-fd32-49e3-bcf2-545da87ae126/storage-initializer/0.log" Jun 11 15:21:11.768299 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:11.768279 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-r269n_e81f5626-fd32-49e3-bcf2-545da87ae126/main/0.log" Jun 11 15:21:12.007258 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:12.007235 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t_5ed0df61-9c23-4536-adad-88bd9aa63409/storage-initializer/0.log" Jun 11 15:21:12.015521 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:12.015435 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-rn56t_5ed0df61-9c23-4536-adad-88bd9aa63409/main/0.log" Jun 11 15:21:12.244389 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:12.244357 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk_2d354704-8045-44aa-8137-bc5d55077b22/storage-initializer/0.log" Jun 11 15:21:12.251646 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:12.251624 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-hz2gk_2d354704-8045-44aa-8137-bc5d55077b22/main/0.log" Jun 11 15:21:12.363679 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:12.363644 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-dhppd_f78d8c8c-1caa-4a1e-a2d1-15879a3e5693/main/0.log" Jun 11 15:21:12.370298 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:12.370280 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-dhppd_f78d8c8c-1caa-4a1e-a2d1-15879a3e5693/storage-initializer/0.log" Jun 11 15:21:19.832931 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:19.832897 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2zl8k_b6572eff-fb22-4baa-b130-995f9284a608/global-pull-secret-syncer/0.log" Jun 11 15:21:19.994246 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:19.994218 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8758w_151bfb0e-d6da-4843-8f17-b1a14999fd3c/konnectivity-agent/0.log" Jun 11 15:21:20.102757 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:20.102691 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-77.ec2.internal_d985ea8e747104fff16cbaa5995c0307/haproxy/0.log" Jun 11 15:21:24.567861 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:24.567829 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-658bf8c56f-c76nr_2bccfb08-b8b0-4eca-ac07-02432f452315/authorino/0.log" Jun 11 15:21:24.626377 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:24.626347 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-n59rn_6d35b04a-0e6d-45d0-82c0-189b14a9cba3/manager/0.log" Jun 11 15:21:24.683522 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:24.683497 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2zlx7_e8ee22a4-188d-43fe-8a49-e5c9bbfa6f4c/kuadrant-console-plugin/0.log" Jun 11 15:21:24.720469 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:24.720439 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h7t6t_a0a2f432-9536-4e09-9aea-6fdc24fdc356/registry-server/0.log" Jun 11 15:21:24.834366 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:24.834239 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tfhct_5d294d1e-eafb-4f15-aa24-bbd4a7194e84/manager/0.log" Jun 11 15:21:26.819513 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:26.819455 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-wq5z5_848c9644-faa6-430e-8cc1-b8b3f451f7bd/cluster-monitoring-operator/0.log" Jun 11 15:21:26.859938 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:26.859886 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-hctsz_d640b8eb-8644-42cc-9303-d8af2af0457d/kube-state-metrics/0.log" Jun 11 15:21:26.876858 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:26.876822 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-hctsz_d640b8eb-8644-42cc-9303-d8af2af0457d/kube-rbac-proxy-main/0.log" Jun 11 15:21:26.899008 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:26.898990 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-hctsz_d640b8eb-8644-42cc-9303-d8af2af0457d/kube-rbac-proxy-self/0.log" Jun 11 15:21:26.987844 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:26.987823 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dvjtj_73a6403f-a8fc-4803-867c-501b5b67eec8/node-exporter/0.log" Jun 11 15:21:27.005940 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:27.005922 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dvjtj_73a6403f-a8fc-4803-867c-501b5b67eec8/kube-rbac-proxy/0.log" Jun 11 15:21:27.023686 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:27.023666 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dvjtj_73a6403f-a8fc-4803-867c-501b5b67eec8/init-textfile/0.log" Jun 11 15:21:27.669454 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:27.669378 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b948c97b5-gp2m9_e50e8c30-4f56-4f29-9332-74049e07838c/thanos-query/0.log" Jun 11 15:21:27.692734 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:27.692697 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b948c97b5-gp2m9_e50e8c30-4f56-4f29-9332-74049e07838c/kube-rbac-proxy-web/0.log" Jun 11 15:21:27.726969 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:27.726927 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b948c97b5-gp2m9_e50e8c30-4f56-4f29-9332-74049e07838c/kube-rbac-proxy/0.log" Jun 11 15:21:27.748209 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:27.748179 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b948c97b5-gp2m9_e50e8c30-4f56-4f29-9332-74049e07838c/prom-label-proxy/0.log" Jun 11 15:21:27.766934 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:27.766909 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b948c97b5-gp2m9_e50e8c30-4f56-4f29-9332-74049e07838c/kube-rbac-proxy-rules/0.log" Jun 11 15:21:27.786194 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:27.786174 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6b948c97b5-gp2m9_e50e8c30-4f56-4f29-9332-74049e07838c/kube-rbac-proxy-metrics/0.log" Jun 11 15:21:30.698983 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:30.698956 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5654b4ffc4-42z52_65bb7535-4585-49ad-bbeb-ff1837efc8b7/console/0.log" Jun 11 15:21:30.723621 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:30.723594 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-g2qng_0b2a36ef-be5f-4768-8925-891462a9549a/download-server/0.log" Jun 11 15:21:31.168723 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.168691 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm"] Jun 11 15:21:31.172164 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.172148 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.174920 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.174893 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cm9qx\"/\"kube-root-ca.crt\"" Jun 11 15:21:31.175963 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.175943 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cm9qx\"/\"openshift-service-ca.crt\"" Jun 11 15:21:31.175963 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.175975 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cm9qx\"/\"default-dockercfg-wv8hg\"" Jun 11 15:21:31.181436 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.181412 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm"] Jun 11 15:21:31.286046 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.286018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-lib-modules\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.286225 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.286071 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7jjb\" (UniqueName: \"kubernetes.io/projected/297c97d0-bffb-4509-abf0-9cc49ebc84d3-kube-api-access-c7jjb\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.286225 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.286097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-sys\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.286225 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.286135 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-podres\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.286225 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.286157 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-proc\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387184 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-lib-modules\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387377 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387203 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7jjb\" (UniqueName: \"kubernetes.io/projected/297c97d0-bffb-4509-abf0-9cc49ebc84d3-kube-api-access-c7jjb\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387377 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-sys\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387377 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387284 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-sys\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387377 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-podres\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387377 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-proc\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387377 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387335 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-lib-modules\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387653 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387454 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-proc\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.387653 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.387459 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/297c97d0-bffb-4509-abf0-9cc49ebc84d3-podres\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.396689 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.396669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7jjb\" (UniqueName: \"kubernetes.io/projected/297c97d0-bffb-4509-abf0-9cc49ebc84d3-kube-api-access-c7jjb\") pod \"perf-node-gather-daemonset-nj2xm\" (UID: \"297c97d0-bffb-4509-abf0-9cc49ebc84d3\") " pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.484559 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.484483 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.613761 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.613734 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm"] Jun 11 15:21:31.615352 ip-10-0-140-77 kubenswrapper[2562]: W0611 15:21:31.615331 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod297c97d0_bffb_4509_abf0_9cc49ebc84d3.slice/crio-11c1d487a31bad0c770edd6063c475c4f505b701fda7426c5decf5a9b7ad7f95 WatchSource:0}: Error finding container 11c1d487a31bad0c770edd6063c475c4f505b701fda7426c5decf5a9b7ad7f95: Status 404 returned error can't find the container with id 11c1d487a31bad0c770edd6063c475c4f505b701fda7426c5decf5a9b7ad7f95 Jun 11 15:21:31.836445 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.836411 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" event={"ID":"297c97d0-bffb-4509-abf0-9cc49ebc84d3","Type":"ContainerStarted","Data":"fb37da5d39d0dcf6aa1dbcdb86e28e4d4dc7b549aacdfdbca6ec57bf5f0d225b"} Jun 11 15:21:31.836840 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.836465 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" event={"ID":"297c97d0-bffb-4509-abf0-9cc49ebc84d3","Type":"ContainerStarted","Data":"11c1d487a31bad0c770edd6063c475c4f505b701fda7426c5decf5a9b7ad7f95"} Jun 11 15:21:31.836840 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.836537 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:31.854791 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:31.854743 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" podStartSLOduration=0.854727646 podStartE2EDuration="854.727646ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 15:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 15:21:31.852800641 +0000 UTC m=+1348.629432364" watchObservedRunningTime="2026-06-11 15:21:31.854727646 +0000 UTC m=+1348.631359358" Jun 11 15:21:32.739121 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:32.739089 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvgk5_94eacb66-220d-4890-90a3-5840d70579ae/dns/0.log" Jun 11 15:21:32.757112 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:32.757088 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvgk5_94eacb66-220d-4890-90a3-5840d70579ae/kube-rbac-proxy/0.log" Jun 11 15:21:32.797659 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:32.797636 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bjhzp_d6f5d097-246f-4602-a126-182e5018d288/dns-node-resolver/0.log" Jun 11 15:21:33.359418 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:33.359371 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-99vwj_8c998f7d-b189-4843-a760-069af193cde6/node-ca/0.log" Jun 11 15:21:34.328870 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:34.328831 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795sl7jz_44715b9a-f62e-4b39-b48d-4dd4c0b761a1/istio-proxy/0.log" Jun 11 15:21:34.441850 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:34.441807 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/1.log" Jun 11 15:21:34.442266 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:34.442182 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/0.log" Jun 11 15:21:34.483129 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:34.483106 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-7xsvk_e89e18ca-c9a2-44d5-9f12-5acd221e3c7e/payload-processing/0.log" Jun 11 15:21:34.507621 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:34.507592 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-868f598b87-7svzh_5c39500a-f1fc-4a5f-b922-1ff7217234b0/router/0.log" Jun 11 15:21:35.210167 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:35.210137 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v95cn_c828ebd1-aa90-4f02-b6c3-c3ecdd97a4b2/serve-healthcheck-canary/0.log" Jun 11 15:21:35.771842 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:35.771816 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6nv4q_595146ac-1555-4472-8500-1f4b84565a05/kube-rbac-proxy/0.log" Jun 11 15:21:35.792557 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:35.792530 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6nv4q_595146ac-1555-4472-8500-1f4b84565a05/exporter/0.log" Jun 11 15:21:35.811974 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:35.811947 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6nv4q_595146ac-1555-4472-8500-1f4b84565a05/extractor/0.log" Jun 11 15:21:37.852323 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:37.852299 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cm9qx/perf-node-gather-daemonset-nj2xm" Jun 11 15:21:38.183042 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:38.182961 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-74d6667c75-8hgrn_f984a2c7-2a44-4d0b-ab25-65db6fb3a47d/nginx/0.log" Jun 11 15:21:38.256250 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:38.256217 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-845d5d9dc5-6djdw_ff927c31-f627-441f-a647-379cea63a16a/maas-api/0.log" Jun 11 15:21:38.306468 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:38.306441 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7d674c68f5-9t56l_eb77ad2d-531c-4a5c-aaee-8eb1e5a59425/manager/0.log" Jun 11 15:21:38.393619 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:38.393592 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh_63aa2c07-839a-4e40-8dfe-06bb7782045e/manager/0.log" Jun 11 15:21:38.413523 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:38.413503 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f8d74d5bf-mtqjh_63aa2c07-839a-4e40-8dfe-06bb7782045e/copy-manifests/0.log" Jun 11 15:21:40.243124 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:40.243093 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5f7555c9dd-mhf6d_ccbae6f2-de45-465d-83bc-bf4fb96a089b/manager/0.log" Jun 11 15:21:45.788563 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:45.788530 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795sl7jz_44715b9a-f62e-4b39-b48d-4dd4c0b761a1/istio-proxy/0.log" Jun 11 15:21:45.874300 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:45.874268 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/1.log" Jun 11 15:21:45.874663 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:45.874645 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/0.log" Jun 11 15:21:45.895593 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:45.895558 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-7xsvk_e89e18ca-c9a2-44d5-9f12-5acd221e3c7e/payload-processing/0.log" Jun 11 15:21:45.910268 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:45.910249 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-868f598b87-7svzh_5c39500a-f1fc-4a5f-b922-1ff7217234b0/router/0.log" Jun 11 15:21:47.014557 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.014525 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795sl7jz_44715b9a-f62e-4b39-b48d-4dd4c0b761a1/istio-proxy/0.log" Jun 11 15:21:47.125606 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.125574 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/1.log" Jun 11 15:21:47.126131 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.126114 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/0.log" Jun 11 15:21:47.157648 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.157625 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-7xsvk_e89e18ca-c9a2-44d5-9f12-5acd221e3c7e/payload-processing/0.log" Jun 11 15:21:47.177317 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.177300 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-868f598b87-7svzh_5c39500a-f1fc-4a5f-b922-1ff7217234b0/router/0.log" Jun 11 15:21:47.576616 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.576541 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-529bv_b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7/kube-multus-additional-cni-plugins/0.log" Jun 11 15:21:47.597491 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.597468 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-529bv_b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7/egress-router-binary-copy/0.log" Jun 11 15:21:47.620700 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.620678 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-529bv_b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7/cni-plugins/0.log" Jun 11 15:21:47.641700 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.641682 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-529bv_b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7/bond-cni-plugin/0.log" Jun 11 15:21:47.665605 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.665569 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-529bv_b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7/routeoverride-cni/0.log" Jun 11 15:21:47.688173 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.688151 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-529bv_b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7/whereabouts-cni-bincopy/0.log" Jun 11 15:21:47.708535 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:47.708513 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-529bv_b02b7dc5-d226-4d1a-97f4-f2c6e4d894d7/whereabouts-cni/0.log" Jun 11 15:21:48.047875 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:48.047843 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fhks6_96157a6e-8a7c-444c-807e-8967dc056298/kube-multus/0.log" Jun 11 15:21:48.184893 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:48.184872 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jxjxg_46e97f49-b264-4a5e-8f66-9ae0ea812103/network-metrics-daemon/0.log" Jun 11 15:21:48.202907 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:48.202883 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jxjxg_46e97f49-b264-4a5e-8f66-9ae0ea812103/kube-rbac-proxy/0.log" Jun 11 15:21:48.776777 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:48.776750 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795sl7jz_44715b9a-f62e-4b39-b48d-4dd4c0b761a1/istio-proxy/0.log" Jun 11 15:21:48.875333 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:48.875300 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/1.log" Jun 11 15:21:48.875628 ip-10-0-140-77 kubenswrapper[2562]: I0611 15:21:48.875611 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-tvsfq_203308f5-42e0-477e-a474-0a9e2d623408/istio-proxy/0.log"